GitHub - jha-naman/treetags: Generate Vim/NeoVim compatible tags for multiple languages

GitHub

7 min read Original article ↗

Add code navigation for multiple languages in Vi/Vim/Neovim.

Leverages tree-sitter to fulfill the goal of supporting multiple programming languages with minimal effort without sacrificing maintainability or performance.

To get a brief overview of what treetags does and how it compares to Universal ctags see here.

More information

Natively Supported Languages

Support for these languages is available out of the box in treetags

Full support with extension fields

  • C
  • C++
  • Go
  • JavaScript
  • Python
  • Rust
  • TypeScript

Refer to Universal ctags documentation for more about extension fields.

Basic navigation support without extension fields

  • Bash/Sh
  • C#
  • Elixir
  • Haskell
  • Java
  • Julia
  • Lua
  • Ocaml
  • PHP
  • Ruby
  • Scala

WASM plugins

Treetags functionality can be extended via WASM plugins. WASM plugins are functionally equivalent to native tag generators, except they are not installed by default and they can be updated independently of treetags releases.

Plugins can be managed using the treetags cli

treetags plugin available # List all known compatible plugins
treetags plugin installed # List all installed plugins
treetags plugin install <NAME> # Fetch and install <NAME> plugin
treetags plugin uninstall <NAME> # Un-install <NAME> plugin
treetags plugin update # Update all installed plugins
treetags plugin help # Print `treetags plugin` usage help

See here for more information on plugin implementation details.

Treetags has support for these languages via user installable WASM plugins.

Full support with extension fields

  • Java
  • Kotlin

Extending treetags language support via tree-sitter tag queries and grammars

Users need to provide two things for treetags to be able to generate tags for languages that do not have a builtin grammar supplied.

  • Precompiled tree-sitter grammar for the language. tree-sitter-langs project is one source with many pre-compiled grammars
  • Tags query file for the language
  • List of file extensions for which the grammar/tag combo is to be used
  • Add an entry to the [[user_grammars]] toml array of the treetags config file

An example config for Kotlin language located at the defualt location of ~/.config/treetags/config.toml is shown below:

[[user_grammars]]
language_name = "kotlin"
grammar_lib_path = "/home/naman/.local/share/nvim/lazy/nvim-treesitter/parser/kotlin.so"
query_file_path = "/home/naman/.config/treetags/queries/kotlin.scm"
extensions = ["kt", "kts"]

Languages with preprovided tags query and extensions

Some languages have tags query and extensions built-in into treetags. Users only need to provide the tree-sitter grammar in that case for treetags to be able to generate tags for that language. Leave the query_file_path empty for these languages to use the tags queries provided with treetags.

  • Gleam

Installation

There are three prerequisites to build treetags

  1. Install Rust and C developmet toolchains on your system

  2. Add wasm32-wasip2 target for rustc

rustup target add wasm32-wasip2
  1. Make sure that rustc uses the clang shipped with wasi-sdk by setting the binCC environment varaible.
export WASI_SDK_PATH=/home/username/play/wasi-sdk-30.0-arm64-linux
export binCC="${WASI_SDK_PATH}/bin/clang --sysroot=${WASI_SDK_PATH}/share/wasi-sysroot"

Building treetags once development setup is complete:

cargo build --release
cp target/release/treetags /somewhere/in/the/PATH/

Recommended Usage

While it is fine to manually invoke treetags to generate tags file for a project, the recommended way is to use the gutentags plugin to manage the tags file. There is a nice write-up on setting up gutentags here, which can be useful for setting things up.

You will have to configure gutentags to use treetags as the tags generator at a minimum in your vim/nvim configuration file.

let g:gutentags_ctags_executable = 'treetags'

Or, if you are using lua for configuration

vim.g.gutentags_ctags_executable = 'treetags'

Generate shell autocomplete scripts

Users can generate completions like:

# refer to your shell documentation for determining the correct path for autcomplete files
treetags completions bash > ~/.local/share/bash-completion/completions/treetags
treetags completions zsh > ~/.local/share/zsh/site-functions/_treetags
treetags completions fish > ~/.config/fish/completions/treetags.fish

Running Integration Tests

Integration tests are built from test cases on demand

cargo build  # Generates test files
cargo test   # Runs all tests including generated ones

Command line options

Use the --help option to see supported command line arguments.

$ target/release/treetags --help
Generate vi compatible tags for multiple languages

Usage: treetags [OPTIONS] [FILE_NAMES]...

Arguments:
  [FILE_NAMES]...  List of file names to be processed when `--append` option is passed

Options:
  ... # Options omitted for brevity

How treetags selects a language

For each input file, treetags picks a language in this order:

  1. --language-force=<lang> — if given, every file is parsed as <lang> (accepts a language name or alias; auto disables it).
  2. Filename patterns — globs matched against the base name, e.g. Rakefile, Gemfile, .bashrc, or *.gemspec. This matches files that have no distinguishing extension.
  3. File extension — e.g. .rs, .py, .rb. Some extensions map to more than one language: .h may be C or C++, and treetags disambiguates by scanning the file for C++ signals (class, namespace, ::, …), defaulting to C when there are none.
  4. #! shebang line — when the name gives no match, the interpreter (python3, bash, ruby, …) selects the language. To keep things cheap, this runs only for files with the executable bit set, unless you pass --guess-language-eagerly / -G, which enables it for every file. (On platforms without an executable bit, shebang detection requires -G.)
  5. Editor modelines — as a last resort under -G, treetags reads Vim modelines (vim: set ft=python:) and Emacs modelines (-*- mode: python -*- and trailing Local Variables: blocks) from the file's head and tail.

Run treetags --print-language <files...> to see which language each file resolves to (or NONE) without generating tags.

Customizing the language map

You can override which extensions and filename patterns map to a language (syntax mirrors Universal Ctags):

  • --map-<LANG>=[+|-]<item> — add (+, the default) or remove (-) a single matcher. <item> is one of:

    • .ext — a file extension (e.g. --map-c=.qc)
    • (pattern) — an fnmatch glob on the basename (e.g. --map-ruby=(Jarfile))
    • %regex% — a regular expression matched against the whole relative path (directory components included), so it can express things a basename glob can't (e.g. --map-c++=%include/.*\.h% treats headers under include/ as C++). Append i or {icase} for case-insensitivity (%…%i), and escape a literal % as \%.

    Repeatable. Regexes take precedence over patterns, which take precedence over extensions. Examples: --map-c=.qc, --map-c=-.h, --map-ruby=(Jarfile).

  • --langmap=<LANG>:<spec>[,<LANG>:<spec>...] — bulk form. <spec> is a run of .ext and (pattern) tokens (e.g. .c.h or (Makefile).mak). A leading + on the spec appends; otherwise it replaces the language's mappings. (Regexes are only available via --map-<LANG>, matching ctags.)

  • --list-maps[=<LANG>] — print the effective extensions and patterns for every language (or just <LANG>) and exit.

What does treetags do

Treetags creates a tags file that vim can use for allowing the user to easily navigate their source code files.

We can quote vim help files to get an idea of what a tag and a tags file are:

What is a tag? It is a location where an identifier is defined. An example is a function definition in a C or C++ program. A list of tags is kept in a tags file. This can be used by Vim to directly jump from any place to the tag, the place where an identifier is defined.

To get a full overview of tags related functionality backed into vim/neovim one can use :help tagsrch in vim or refer to the online docs

This is similar to what Universal ctags and other verions of ctags do. The ctags versions are much more mature and battle tested than treetags. Universal ctags in particular also has support for many more languages and is actively maintained.

Treetags differs from these in two ways technically. First is that it uses tree-sitter for parsing code. Second is that treetags is multithreaded and can parse multiple files simultaneously. Another important difference is that treetags is primarily maintained by a single person.

How treetags compares to LSP

Refer to this issue to see how tags compare to LSP.