Substack to Local html and to epub

Problem

To convert the public posts of a substack blog into epub.

Solution

By using sbstck-dl.

Step 1: first install the Go language. The easiest way which would download, install the language, and then configure the PATH automatically.

In PowerShell (windows) > winget install GoLang.Go

Step 2: verify it
> go version

Step 3: download the sbstck-dl-win-amd64.exe file from the above mentioned sbstck-dl repo. The method of installation provided in the repo installs an earlier version.

Step 4: Better to create a new directory with the exe file in it and then run the following command .\sbstck-dl-win-amd64.exe download --url https://example.substack.com --download-images --create-archive

This will download the entire public blog at example.substack.com, download all the files in html format (sbstck-dl has options to download individual posts as well as into md or text formats, instead of html), and also download the images in a separate folder. These html files can then be read in the browser directly. The create-archive option creates an index.html file which can then be used for TOC (table of contents) for the entire blog.

Step 5: To convert the entire stack of html files into an epub, there are two ways: either use ebook-convert (you get that with Calibre) or by using pandoc. Both are available in scoop and can be installed scoop install calibre or scoop install pandoc.

Step 6: For text-heavy substacks, it is better to ignore the images and just have the texts. first create a local file no-images.lua with the following text

function Image(el)
  return pandoc.Str("[image]")
end 

Step 7: Because of some windows powershell glob issues, you will have to run the following command in Powershell (this needs the index.html, all the blogpost html, the images folder, and the no-images.lua files in the same folder where the command is being run).

pandoc (Get-ChildItem *.html | Where-Object Name -ne "index.html" | Sort-Object Name).FullName -o Name_of.epub --lua-filter=no-images.lua --metadata title="Name_of"

or in cmd.exe

pandoc *.html -o Name_of.epub --lua-filter=no-images.lua --metadata title="Name_of"

Happy Reading!