AllMyToes is both an executable binary that can be run, and a library that can be used in Rust programs.

Installing the command-line executable

Assuming you have Rust/Cargo installed, run this command in a terminal:

cargo install allmytoes

It will make the allmytoes command available in your PATH if you've allowed the PATH to be modified when installing Rust. cargo uninstall allmytoes uninstalls.

Adding `allmytoes` library as a dependency

Run this command in a terminal, in your project's directory:

cargo add allmytoes

To add it manually, edit your project's Cargo.toml file and add to the [dependencies] section:

allmytoes = "0.5.1"

The allmytoes library will be automatically available globally. Read the allmytoes library documentation.

Back to the crate overview.

Readme

AllMyToes

Provides thumbnails by using the freedesktop-specified thumbnail database (aka XDG standard).

Thumbnails are shared with other programs via a common cache.

The (expensive) creation of thumbnails is only done if a thumbnail has not been created yet or if the original file has changed.

AllMyToes can create thumbnails for many image formats. (See “Decoding” support of the image crate.)

Additionally, AllMyToes can create thumbnails for other kind of files using other programs to create the base image. (See section Provider.)

AllMyToes is both, a non-interactive program for the shell and scripts, and a Rust library. It's abbreviated “AMT” (or “amt”) in code or structured data.

[[TOC]]

Usage

Call allmytoes with a file as parameter. The program will print the path to a thumbnail to stdout.

> allmytoes some_image.jpg
/home/me/.cache/thumbnails/large/b7931d1d6e0439c1a6e2e6b02c5b21a6.png

If the thumbnail already exists, AllMyToes will just return the path to the existing thumbnail file. If not, AllMyToes will create the image’s thumbnail first.

Thumbnail Size

The latest freedesktop.org specification (0.9.0) defines four different thumbnail sizes: normal, large, x-large, and xx-large with maximum edge lengths of 128 px, 256 px, 512 px, and 1024 px respectively.

By default, AllMyToes returns the path to the large (256 px) thumbnail. The size to be returned can be chosen with the -s (--size) option by giving a value out of {n, l, x, xx}.

For example, to get a xx-large thumbnail, the call would look like this:

> allmytoes -sxx some_image.jpg
/home/me/.cache/thumbnails/xx-large/ad0779df58de36f038bdc4040a322bfe.png

Small Input Images

If the input image is smaller than the requested thumbnail size, the thumbnail will have the same dimensions as the input image. A thumbnail will never be bigger than the original image.

If a thumb for the requested size does not already exist, AllMyToes will check if the input image is smaller than the requested thumb size. If so, AllMyToes will determine the smallest thumb size that still covers the input image dimensions (is bigger or equal than the input image). When that “feasible” thumb size is different from the requested thumb size, AllMyToes will start the job all over with that smaller, “feasible” thumb size. AllMyToes does that to avoid unnecessary, duplicated thumbnails for small images.

That implies that for small input images, the returned thumbnail path might not correspond to the requested thumb size, because creating and keeping thumbnails for the (bigger) requested size would be a waste of computation time and disk space.

If – for whatever reason – one must get a result path to the thumbnail of the requested size, one can use the -F (--force-size) switch. However, one should do that only if there is a good reason.

Show Extensive Information

Calling AllMyToes with the --extensive (or -e) option makes it print out more information than just the path to the thumb. The provided information is

The path to the thumb
The URI-hash of the input file that is used for the XDG-thumb name
All metadata (key-value pairs) stored in the thumb (as “tEXt” entries)

It does not matter if the thumb did already exist or if it has just been created by AllMyToes.

All data is printed as a list of key-value pairs, each tuple in one line. The key is separated from the value by “: ”. Key-value pairs which show metadata from the thumbnail file are preceded with a colon and are printed in alphabetical order.

Example:

$ allmytoes --extensive some_image.jpg 
Thumb path: /home/dude/.cache/thumbnails/large/c000276b8378a33e16e3fac005eebc02.png
Thumb hash: c000276b8378a33e16e3fac005eebc02
:Software: allmytoes
:Software::Version: 0.1.0
:Thumb::Image::Dimensions: 4608x3456
:Thumb::Image::Height: 3456
:Thumb::Image::Reorientation: Rotated 180 degrees, not flipped
:Thumb::Image::Width: 4608
:Thumb::MTime: 1693737198
:Thumb::Mimetype: image/jpeg
:Thumb::Size: 7777807
:Thumb::URI: file:///path/to/some_image.jpg

What meta-data is available in the thumb of course depends on the software which created the thumb. So, the lines starting with a colon (:) may vary.

Logging

By default, AllMyToes will print warnings and errors to stderr. A different log-level can be set via the environment variable RUST_LOG. For example, do a

export RUST_LOG=trace

to get all log entries, down to the most detailed “trace”-level.

Provider

AllMyToes can create thumbnails for non-image files if a “provider” is defined. A provider is a shell-command or a script that provides an image for a certain set of MIME types.

AllMyToes has inbuilt support for some file formats:

Format	Dependencies
PDF	(`evince-thumbnailer` or `magick`) and (`exiftool` or `pdfinfo` or `gs`)
Postscript	(`evince-thumbnailer` or `magick`) and (`exiftool` or `gs`)
SVG	`inkscape` or `magick`
various video formats	`ffmpeg` (and optionally `ffprobe` & `magick` & `bc` for more complex thumbs) and (`ffprobe` or (`mediainfo` and `bc`) or (`exiftool` and `bc`))

The exact MIME-types and the provider definitions can be found in /conf/provider.yaml.

Defining Provider

Users can use their own provider configuration in a YAML file to add support for more MIME types or to implement more fancy thumbnails.

To do so, copy /conf/provider.yaml to ~/.config/allmytoes/provider.yaml (or $XDG_CONFIG_HOME/allmytoes/provider.yaml if $XDG_CONFIG_HOME is defined) and adapt it. You can also use the -p (--provider-config-file) option to explicitly choose another provider configuration.

❗ Be aware that the provider-feature is pretty new and the format of the provider configuration will likely change here and there before it stabilizes.

The provider configuration YAML file has a list of dictionaries on the top level. Each list entry defines a provider for one or more MIME types. For each entry, there are two mandatory and two optional keys.

Key	Mandatory	Short description
`mimes`	Yes	List of mime types, handled by the provider definition
`commands`	Yes	List of commands providing the thumbnail, processed top down until one succeeds
`meta`	No	Dictionary mapping thumb-meta-keys to a list of commands to provide them
`revision`	No	integer to track the version of the provider definition

Key `mimes`

The mime type list defines for which mime types the provider is executed. MIME types are specifies as <type>/<subtype>, like video/x-matroska. If AllMyToes is called for a file for which no provider can be found, the MIME type is printed in an error message. MIME types for which a provider is found are printed in a debug message and also annotated as meta-data in the thumbnail itself. (Which you can see if you use the --extensive (-e) option.)

Key `commmands`

Each provider-entry must have a list of commands. Commands are shell commands (or scripts) that take the input file and return an image to AllMyToes. AllMyToes takes that image and creates the XDG-compliant thumbnail from it. Commands provide that image by copying it to a defined location.

When one of the MIME types from the mimes list matches the input file, AllMyToes executes the first command from the commands list. If the command succeeds (exit code 0), AllMyToes stops and uses the thumbnail which has been provided by that command.

If the command does not succeed (exit code ≠ 0), AllMyToes tries the next command, and so forth.

This allows to have commands which use different programs to provide the thumb. If a user does not have the programs used in the first command, another option to calculate the thumb may work. If a command returns 0 but still does not provide a valid thumb, AllMyToes returns with an error.

A command provides the thumb by copying it to a certain path in a temporary directory, which is deleted by AllMyToes afterwards.

Commands need certain information to do their job, for example the input file and the path to copy the thumbnail to. Therefore, commands can use certain variables.

Variable	Content
`%(file)s`	The full path to the input file
`%(size)s`	The size in pixels that the longer edge should ideally have.
`%(outfile)s`	The path to the temporary file where the thumbnail shall be copied to
`%(tmpdir)s`	The temporary directory that contains `%(outfile)s`, and that can be used other temporary files that might be needed during the thumb-creation

If the created image (at %(outfile)s) is bigger than %(size)s, AllMyToes will scale the image down accordingly. AllMyToes also does other necessary conversions like turning the image into an 8-bit per channel RGBA PNG image and add all necessary meta chunks. So, the commands can provide their thumbnails in any supported image format. AllMyToes will take care to turn the image into a freedesktop.org-compliant thumbnail afterwards.

Key `meta`

freedesktop.org-thumbnails have “metadata chunks”, meta-data attributes in key-value pairs. (See explanation of meta-data chunks.) Providers can also provide such meta-data entries. Some are even recommended by the standard (Thumb::Document::Pages for paper-oriented documents and Thumb::Movie::Length for the length of video files in seconds).

The meta key in the provider specification contains a dictionary, mapping meta-data keys to lists of commands. For each key, the list of commands are executed like for the thumb image top down until one of them exits successfully with 0. The value for the meta-chunk has to be returned via stdout.

With the current implementation, the thumb-generation fails if none of the commands for a meta-chunk is successful. The commands for a meta-chunk can use the same variables as the commands for the image creation.

The specification of meta-chunks is optional.

Key `revision`

The revision key is intended as a “version” for the provider. It's optional and as of now, it does not have any effect other than being added as a meta-chunk to the thumbnail.

The idea is that this revision number can be used in the future to re-create thumbs if a provider is available with a higher revision.

Notes on commands

One should consider a few things for both, commands for the images in the commands key, and for commands under a meta key.

First, commands can be multi-line shell scripts, simply by using multi-line strings in the YAML file. This example for an SVG-provider shows how multi-line commands may look like:

- mimes:
  - image/svg+xml
  commands:
    - |
      which inkscape || exit 1
      w=$(inkscape -W "%(file)s" | sed 's/\..*//')
      h=$(inkscape -H "%(file)s" | sed 's/\..*//')
      test $w -gt $h && size_arg="-w %(size)s" || size_arg="-h %(size)s"
      inkscape --export-area-page --export-type=png $size_arg -o "%(tmpdir)s/outfile.png" "%(file)s" && \
        mv "%(tmpdir)s/outfile.png" "%(outfile)s"
    - |
      convert -background none -resize %(size)sx%(size)s "%(file)s" "%(tmpdir)s/outfile.png" && \
        mv "%(tmpdir)s/outfile.png" "%(outfile)s"

Secondly, commands should be POSIX shell compatible, not - for example - depend on BASH. This assures that the provider commands will work on most machines and accounts.

Third, commands should fail with a return code > 0 if some environment preconditions are not fulfilled. Keep in mind that a pipe of commands exits with the return code of the last command, no matter if an earlier command did fail.

So, it's a good idea to chain programs with && or to extend program calls with || exit 1 to make a provider command fail properly. The availability of required programs that are used in pipes can for example be tested with which <program> || exit 1.

Also remember to quote the input file (like "%(file)s") as the path may contain spaces. The %(outfile)s (and the %(tmpdir)s) should not contain any spaces, but it doesn't hurt to also quote them.

Example

Let's look at an example and specify a provider that provides thumbnails for PDF and postscript files. Both can be turned into a thumbnail image with the same command, which is why it's possible to handle both in one provider.

Be aware that the real default configuration for AllMyToes uses a different provider definition for these formats to implement more specific meta-chunk commands. This definition serves only as an example.

PDF files have a MIME type application/pdf, postscript files have application/postscript.

- mimes: [application/pdf, application/postscript]
  commands:
    - evince-thumbnailer -s %(size)s "%(file)s" %(outfile)s
    - convert -thumbnail %(size)sx%(size)s -background white %(file)s[0] PNG:%(outfile)s
  meta:
    'Thumb::Document::Pages':
      - |
        which exiftool || exit 1
        exiftool "%(file)s" | awk '/^Page Count/ {print $4}' | sed 's/[^0-9]*//g'
      - | 
        which gs || exit 1
        gs -o /dev/null -sDEVICE=bbox "%(file)s" 2>&1 | grep HiResBoundingBox | wc -l
  revision: 1

First, the MIME types for which this provider shall be used are defined.

The commands section first tries to create the thumbnail (in %(outfile)s) with evince-thumbnailer. If evince-thumbnailer exists, the task of providing the image is done and the second command is not executed.

When evince-thumbnailer is not available, the first command fails and AllMyToes will execute the second command which uses convert from ImageMagick. If also convert is not available, the thumb creation has failed and AllMyToes returns with an error.

The evince-thumbnailer is tried first because it's faster and we prefer that one.

If one of the two commands was successful, AllMyToes evaluates the meta section. In this example, we have only one meta-chunk with the meta-key Thumb::Document::Pages. This meta-chunk has also two commands and again, AllMyToes will first try the first and execute the second one only if the first one fails.

The first command depends on exiftool, the second depends on Ghostscript (gs). Because both of them are used in a pipe, we first check if they exist explicitly and exit with 1 otherwise. The example shows how a multi-line script can be used as a command.

Both of the metadata commands return the value for the Thumb::Document::Pages metadata chunk as output on stdout.

Check the default provider configuration for more examples.

How is this useful?

My personal motivation was the use of thumbnails for image previews in terminal-based file managers like joshuto and ranger. But AllMyToes can be useful in any situation where one wants to show a size-limited image in scripted environments, maybe as icons in desktop notifications, other kind of image-overlays in terminal-based applications, or desktop widgets; wherever loading full sized-images would consume unnecessary time and CPU power.

Installation

Linux X86-64 Binary (Experimental)

Download the latest release as binary from the release page. Make the downloaded file executable (chmod +x allmytoes) and place it in a directory which is part of your PATH (for example mv allmytoes ~/.local/bin).

Latest release from source via `cargo`

cargo install allmytoes

Bleeding edge from source via `cargo`

cargo install --git https://gitlab.com/allmytoes/allmytoes.git

From repo clone

git clone https://gitlab.com/allmytoes/allmytoes.git
cd allmytoes
cargo build --release

The built binary will be at target/release/allmytoes.

Some Specs

Freedesktop.org Thumbnail Specification: AllMyToes strives for compliance to the thumbnail freedesktop.org specification. All relevant parts of the standard should be implemented by now.
AllMyToes will not create the cache directory ($XDG_CACHE_HOME or ~/.cache). In case the cache directory does not exist, AllMyToes will terminate with an error. (Might be changed later.)
AllMyToes will create the thumbnail directory beneath the cache directory and the size-specific subdirectories if they do not exist.
AllMyToes will only accept regular files and symlinks to regular files as input. Directories may get supported in the future. There are no plans for other file types.

Thumbnail Meta-Data Chunks

Thumbnails - all in PNG format - have certain meta-data entries, each of them a PNG “tEXt” chunk with a key and a value. The freedesktop.org thumbnail standard describes some mandatory and some optional keys. (See the “Thumbnail Creation section” in the standard.) AllMyToes implements all of them. The following table lists all meta-data entries created by AllMyToes.

Key	XDG standard	Description
`Thumb::URI`	mandatory	The URI of the thumb's source file.
`Thumb::MTime`	mandatory	The mtime of the thumb's source file. (Functionally required.)
`Software`	optional	The name of the software that created the thumb. Also described by the PNG standard. This is always set to `allmytoes`.
`Software::Version`	-	The version of AllMyToes that created the thumb.
`AMT::ProviderRevision`	-	For thumbs from providers (non image sources), the revision of the provider as given in the config.
`Thumb::Size`	optional	The size of the thumb's source file in bytes.
`Thumb::Mimetype`	optional	The mimetype of the thumb's source file.
`Thumb::Image::Width`	optional	The width of the source image in pixel. ^†
`Thumb::Image::Height`	optional	The height of the source image in pixel. ^†
`Thumb::Image::Dimensions`	-	A string `<width>x<height>`, describing the dimensions of the thumb's source image. ^†
`Thumb::Image::Reorientation`	-	A textual description of the rotation and flipping done my AllMyToes on creation of the thumb based on the EXIF data of the thumb's source image. Only if the source of a thumb is an image format with EXIF data that contains an `Orientation` flag.
`Thumb::Document::Pages`	optional	The number of pages (only for PDF and postscript files).
`Thumb::Movie::Length`	optional	The length of a video in seconds (only for video files).

† Only for image input files

Roadmap

Major milestones planned:

Comply to mandatory requirements from the freedesktop.org specification and have basic robustness (see %1). AllMyToes will go to version 0.1.0 then.
Support for non-image file formats (videos, fonts, documents, ...) by configuring other programs providing previews. (#13)
Support for shared repositories.
Support for other thumbnail repository locations depending on configured path patterns. (Not part of the freedesktop.org standard.)

Testing

Few things are tested by standard Rust unit test. Those tests can be run with cargo test.

The majority of AllMyToes’ functionality is tested with end-to-end tests by a BDD test framework, Python Behave. Those tests use real images on the file system and an allmytoes debug build, started as a sub-process.

The BDD tests live in the test sub-directory. The test-definitions can be found as feature-files in test/features.

Running the BDD tests

Have a Python >= 3.10 environment with the required dependencies. E.g., use a virtualenv and then pip install -r test/requirements.txt.

Have a debug build of AllMyToes. target/debug/allmytoes will be the binary under test.

Then, to run the tests, cd into the test directory and run behave.

Notes on missing Tests / untested things

evaluation of XDG_CACHE_HOME
MIME type probing for wrong and missing file extensions (if this fails on CI, check if the MIME-db is installed on the used docker img)
--force-creation
Return error when input file is not readable, no matter if valid thumb exists.
Updating existing but outdated thumbs
Error on non-existing, non-readable, not-decodable input-file
Deny to process files which are neither a regular nor a symlink to a regular file
All tEXt meta-data (mtime and URI are tested implicitly, of course)
#61

You may prefer to use this instead of AllMyToes

Tumbler is a much more mature and feature-rich tool for the same purpose, but needs to run as a service and uses DBUS for communication.

I did not use Tumbler as I wanted to have something more simple for my scripting.

AllMyToes as a Rust Library

AllMyToes can be used as a Rust library to obtain a thumbnail for a given image. You can find the crate on crates.io.

AllMyToes has a very small interface, easiest to explain by example:

use std::path::Path;
use allmytoes::{AMTConfiguration, AMT, ThumbSize};

fn main() {
    // The configuration for allmytoes.
    // Usually, the defaults are fine.
    // The fields correspond to the `allmytoes` options.
    let configuration = AMTConfiguration::default();

    // An instance of allmytoes that can be used to provide thumbnails.
    let amt = AMT::new(&configuration);

    // The file for which we want a thumbnail as a `std::path::Path`.
    let input_file = Path::new("/tmp/image.jpg");
    
    // The size of the thumbnail we want.
    let thumb_size = ThumbSize::Large;

    // Get a thumbnail.
    // `AMT::get` returns `Result<Thumb, ToeErrorType>`,
    // where `Thumb` provides the path and other data about the thumbnail.
    match amt.get(
        input_file,
        thumb_size,
    ) {
        Ok(thumb) => {
            println!("The thumb is here: {}", thumb.path)
        }
        Err(error) => println!(
            "Error '{:?}' occurred when trying to provide the thumb. ({})",
            error,
            error.msg(),
        ),
    }
}

License: MIT

AllMyToes is free software and licensed under the MIT License.

See the LICENSE file in the project root for details.

Installing the command-line executable

Adding allmytoes library as a dependency

Adding `allmytoes` library as a dependency