Dotbot to Chezmoi

Dotbot to Chezmoi

Every software developer likes to have their own custom configuration for all their tools and most tools are configured using “dotfiles”, hidden files prefixed by a . character. I’d been using dotbot to configure and sync my dotfiles across multiple machines and configure new work laptops but it had been missing a lot of features and was quite cumbersome to work with. Not only did I regularly need to manage the git repo, having to commit changes and push to my GitHub remote every single time I made any changes, I was also regularly fiddling with annoying YAML files any time I wanted to add or remove a new dotfile. It also required that I install dotbot into the repository as a submodule which also extended to any other directories I wanted to sync between installations. Managing submodules is quite annoying since you need to manually update them when there are upstream changes in the submodule. This meant that if I wasn’t touching the repo at all for a while, the submodules would be completely out of date which meant outdated plugins and an outdated dotbot. It also didn’t have many features for cross-platform management such as running certain scripts only in a MacOS or Linux machine. This wasn’t critical to my needs since I always use a MacBook to code generally but I would at least like the option. Say I was using a Linux server that I was using to host services and apps, I could customise the shell to my liking and create a familiar environment. Dotbot is also just a Python project that doesn’t install well as a shell program that can be run anywhere on your system. I would have to go to my dotfiles repo every time to make any changes.

Enter chezmoi, an extremely powerful configuration and dotfiles manager that uses git under the hood and has all the features I felt were missing in dotbot. Since it also installs as a globally available script rather than only in the repo, I can add, edit and remove files from wherever I please when I feel like it. It also has options to automatically commit and push files to my remote repository every time I edit and save my dotfiles. Configuration is extremely easy and very powerful and it uses file name prefixes rather than YAML configs to define the actions to be taken on your files. You can even add templates to have parts of scripts run or place files only if conditions are satisfied. It can even fill in env variables from password managers like 1Password or BitWarden. While trying to figure out ways to move my setup, I found that someone had already asked how they could migrate from a symlinks based setup to chezmoi in the chezmoi GitHub issues and the maintainer provided some very helpful steps.

Migrating my dotfiles

Before I began the migration process, I ensured all the current files in my dotbot based dotfiles repo was up-to-date and created a separate branch to make sure the old code was still available somewhere. No more dotbot anymore for me.

To begin the process, I installed chezmoi using brew with brew install chezmoi and ran chezmoi init to initialise the tool. It creates a new directory and git repo at ~/.local/share/chezmoi. Traditionally, if you were starting fresh without ever having created a dotfiles repo, you could add a remote repo at https://github.com/<username>/dotfiles to where it would push the files. chezmoi would automatically use this directory in the future for commands like init on a new machine where you could simply supply your GitHub username and it would pull the whole project directly. But since I already had a dotfiles repository, I used the steps above and cloned my existing git repo to ~/.local/share/chezmoi. Then, I created a .chezmoiroot file at the top of the directory.

.chezmoiroot
home

This tells chezmoi that rather than use the root of the directory to store the files that it should store everything within this newly created home directory within the repo. This leaves the root of the directory free to manage other files and the chezmoi can sync everything in the home directory correctly. For eg. I store my iterm config within my dotfiles too which makes it easy to sync even my terminal settings between systems.

Now, to add any new files for chezmoi to sync, I’d run the following command.

Terminal window
chezmoi add ~/.zshrc --follow

Since dotbot installed all my dotfiles as symlinks to an instance of my dotfiles repo, I needed to provide the --follow option to chezmoi add since chezmoi needs to follow the symlink to the contexts of the file. without that option, it would simply add the link itself which we don’t want. This creates a new file at ~/.local/share/chezmoi/home/dot_zshrc and from here on, any changes you make to that file can be synced to ~/.zshrc by running chezmoi apply. chezmoi also lets you see what changes will be applied to your local dotfiles by running chezmoi diff.

To test things out, I removed the symlink currently at ~/.zshrc and ran applied the changes.

Terminal window
rm ~/.zshrc
chezmoi apply

I followed these steps for all my files in my current dotbot repo such as my vim config, gitconfig. helix-config etc etc.

I have to admit, removing the older symlinks and the files from my dotbot repo gave me a strong sense of satisfaction haha!

Laughing man persona

Migrating my oh-my-zsh plugins

As mentioned earlier, I’d decided to add all my oh-my-zsh plugins as submodules to my dotfiles repo which was a very annoying requirement. chezmoi instead can fetch the latest artifacts from the git repos of these plugins and extract them into the requisite custom ZSH directory. This is done by creating a .chezmoiexternal.toml file in the home directory that contains the details of where to fetch the files and where to place them.

home/.chezmoiexternal.toml
[".oh-my-zsh/custom/plugins/zsh-syntax-highlighting"]
type = "archive"
url = "https://github.com/zsh-users/zsh-syntax-highlighting/archive/master.tar.gz"
exact = true
stripComponents = 1
# ...

This tells us that the zsh-syntax-highlighting archive from that URL on GitHub needs to be fetched and extracted to ~/.oh-my-zsh/custom/plugins/zsh-syntax-highlighting directory. More options can be explored here. It’s also easy enough to fetch individual files by changing the type and setting the URL correctly which is how I fetch all my preferred themes for the helix text editor that I like to use for quick text editing.

Migrating my scripts

On a Mac machine, I’d like to install all my choice programs with brew. The best way is to use chezmoi templates. All scripts in the chezmoi directory will run in alphabetical order and we can prefix the files so they can install under very specific conditions when you run chezmoi apply. I only want my installation script to run if I make any changes such as changing the brew casks or taps in the shell script.

It is highly recommended that all scripts that you add here are idempotent i.e. after you run the scripts once, they should always return the same result. If all the packages are installed already, they should just not do anything. This means there should be no side effects to any of your scripts so it is best to strictly avoid things like copying files to relative paths and such.

To make sure the brew scripts only run on a Mac, we can wrap the brew installation within a chezmoi template string with the file extension as .tmpl like so.

home/run_onchange_install-packages.sh.tmpl
{{- if eq .chezmoi.os "darwin" -}}
#!/bin/bash
echo 'Installing packages using Brew'
brew bundle --verbose --file=/dev/stdin <<EOF
tap "homebrew/bundle"
tap "homebrew/core"
cask "iterm2"
# ...
EOF
{{ end -}}
# Check if oh-my-zsh is installed
OMZDIR="$HOME/.oh-my-zsh"
if [ ! -d "$OMZDIR" ]; then
echo 'Installing oh-my-zsh'
/bin/sh -c "$(curl -fsSL https://raw.github.com/robbyrussell/oh-my-zsh/master/tools/install.sh)"
else
echo 'Updating oh-my-zsh'
/bin/zsh -c "omz update"
fi

All text within the template tags {{- if eq .chezmoi.os "darwin" -}} and {{ end -}} will only run if chezmoi detects that the machine OS is darwin based i.e. a Mac. I can add a separate condition to install my packages using apt or apt-get if I am running this on a linux machine by adding a section that checks if the OS is Linux using the tag {{ else if eq .chezmoi.os "linux" }}.

Installing oh-my-zsh can be platform agnostic entirely and doesn’t need to be wrapped in the tag. We check whether omz is installed already by checking the existence of the ~/.oh-my-zsh directory and not installing it if it already exists.

Final steps

The final thing I wanted to do was commit to the repo abd oush the new commit to the remote every time I edit a file in the chezmoi directory with chezmoi edit --apply so the last thing was to add the following to chezmoi’s own configuration. In the same vein, we can make sure that chezmoi always sets up a chezmoi configuration file every time it is installed on a new system. We can do this by turning the chezmoi configuration itself into a template!

home/.chezmoi.toml.tmpl
progress = true
[git]
autoCommit = true
autoPush = true

With this simple config change, chezmoi will autocommit the repo with a commit message stating the name of the file that was changed and automatically pushing to the remote repository. Some people might think this is dangerous but I highly doubt I’m making any permanently breaking changes when I make changes in the chezmoi directory. At worst I accidentally delete a file. It’ll be fine.

Conclusion

And that’s the whole migration process from dotbot to chezmoi. The potential that chezmoi unlocks is really cool for configuration management and I’m looking forward to experimenting with it a ton more so I can have unified and consistent environments across all my machines. From here, I can build a configuration for Linux machines so it installs all the tools using apt or apt-get depending on the OS instead of brew. I can even add conditions to only install GUI software like Chrome, Firefox, Insomnia and the like based on a prompt that would ask me so that I can skip those on server only machines. Installing chezmoi and getting my dotfiles on other machines is as easy as running a single command.

Terminal window
sh -c "$(curl -fsLS get.chezmoi.io)" -- init --apply aniforprez

My own settings are at my dotfiles repo.

Comments