cliffs, a small CLI for remembering what your scripts to rule them all are for

Project I’ve worked on at Azavea really frequently use the Scripts to Rule Them All (STRTA) pattern. Using STRTA is great once everyone is really used to what all of the scripts are for — provided no one has done anything weird, it’s much easier to get spun up in a new project with scripts/setup and scripts/update than with a bunch of bespoke, one-off commands relying on whatever each project is calling a “build tool.”

However, a few weeks ago, someone pointed out that while that’s nice for everyone who is used to STRTA, if the scripts reach a point where no one can remember well enough the differences between setup, bootstrap, and update, the scripts can get in the way. You can of course run ./scripts/setup --help, ./scripts/bootstrap --help, and ./scripts/update --help provided your friendly colleagues remembered to include a --help option, but that ends up being a lot of typing to do every time.

Also, over time, the list of available scripts tends to grow much more often than it shrinks. That’s how in Raster Foundry we wound up with sixteen scripts, most of them non-standard. Some of them aren’t even useful / don’t even work anymore!

Something I noticed a bit ago is that some of our projects also have nice tables describing the scripts. I thought: can you use semi-structured READMEs (a bunch of markdown text with a table with special column names) and semi-structured scripts (executable text files that 🤞🏻 print helpful information when called with --help) to show information about what scripts are for in a single interactive location?

It turns out you can!

Then an attentive and conscientious human could use that information never to be confused about what scripts are for and to have an up-to-date view of all the scripts that are still around.

I think I probably poisoned my brain and I’m now seeing every problem as a terminal UI problem, but I think it was worth it to learn a fun new tool.

cliffs

cliffs is a small CLI that does very little, but the little it does is pretty useful. If you have a README with specially named columns and a scripts directory named scripts, running cliffs will get you a nice little interactive session for exploring what each thing does:

In short:

• run cliffs in a directory containing a README.md file and a scripts/ directory
• ⬆️/⬇️ to preview the --help text for a script
• or you can just look at descriptions if they’re sufficient

I think you could use cliffs either to help you find out what scripts are for if you’ve forgotten or to audit your scripts to make sure they all understand what the --help flag does. While frameworks like Python’s click or oclif in Node.js will handle --help for you, if you’re hand-rolling bash scripts, it’s much easier to forget.

You can install cliffs with nix as:

nix-env -i -f https://github.com/jisantuc/cliffs/archive/refs/tags/v0.0.1.zip

Someday there will be a container image, but I don’t actually know how to turn my default.nix into a container yet 🤷🏻‍♂️.

cliffs is built on…

brick

brick handles the terminal UI interactions. brick was a delight to get started with and felt like writing The Elm Architecture for a terminal program. As someone whose brain breaks every time I have to think about frontend layout, getting live in a land of pure functions from state to UI was really nice. Also, the examples directory was really effective as documentation.

cmark-gfm

cmark-gfm stands for “CommonMark GitHub-flavored markdown”. It’s the library responsible for turning raw text into something with some structure. It’s oriented around nodes and their children, which means for any sort of search for a specific kind of element, we just have to walk a tree and find a node matching some predicate. It claims to be very fast, which is nice, but didn’t really come up for the READMEs I tested on. If you can describe in words the relationships between the things you’re looking for — e.g. “the text in the second column of the row” — it’s not super difficult to guess the relationships between nodes that you end up with after parsing with cmark-gfm.

A surprising bonus is that parsing can’t fail — note that the return type for commonmarkToNode doesn’t return an Either or Option or IO of Node, just a Node. I hadn’t considered this before, but the reason for this is that there’s no such thing as an invalid CommonMark document — the CommonMark spec even notes that

nothing in Markdown counts as a “syntax error”

I hadn’t considered this previously and it was cool to learn something about something I use almost every day from the return type of a parser.