dtox - DTO Generator

This tool automates the generation of source code and other artifacts from a set of Data Transfer Object (DTO) definition files. It uses a template-based approach, allowing for flexible code generation for any language or framework.

The generator discovers DTO files, processes templates by replacing placeholders with DTO-specific information, and handles both per-DTO file generation and aggregation of content into single files.

Installation

dtox is now distributed as a regular Rust CLI in addition to the Docker image. If you already have Rust installed, you can install the latest tagged release on macOS, Windows, or Linux with a single command:

cargo install --locked --git https://gitlab.com/hti-oss/rxdatasets/dtox.git --tag 0.7.0

Drop the --tag flag if you prefer tracking master, or add --force to upgrade an existing installation. Helper scripts live in scripts/install.sh (Bash) and scripts/install.ps1 (PowerShell) if you would rather not memorize the cargo install flags.

See INSTALL.md for platform-specific walkthroughs (Rust/Rustup prerequisites, PATH updates, and verification steps). Once installed, run dtox --version to verify the binary is on your PATH.

Usage

Here is a summary of the command-line options, which you can also view by running dtox --help.

How It Works

The generator operates by discovering DTOs, loading configuration, and processing templates.

DTO Discovery

The tool finds DTO definition files in the specified --input-dir.

• File Naming: DTO files must follow the pattern [name].v[version].[extension]. For example, product.v2.dto or order.v1.proto.
• File Content: Each DTO file must contain a line specifying its minor version, like minor version 3. Suggestion is to use a format-specific comment, such as // minor version 3 for protobuf
• Selective Processing: You can optionally provide a file via --list-file (e.g., dtos.lst) containing a newline-separated list of DTO filenames to process. If this is used, only the files listed will be processed - and in that order.

Configuration Files

You can provide DTO-specific values for placeholders using special CSV files in the input directory.

• File Naming: Configuration files must be named __[placeholder_name]__.csv. For example, a file named __csharp_namespace__.csv will provide values for the __csharp_namespace__ placeholder.

• File Format: The CSV file should contain two columns: the DTO identifier and the value. The DTO identifier is [name].v[version]. Comment lines or lines that are not valid CSV are ignored.

  # C# Namespaces for DTOs
  product.v2,MyCompany.Products.V2
  order.v1,MyCompany.Billing.V1
  

Template Processing

The tool processes templates found in the --template-dir (which defaults to --input-dir). There are two main processing modes.

1. Per-DTO Generation

For each discovered DTO, the tool copies all subdirectories from the template directory into a temporary location. It then performs placeholder replacements on all filenames and file contents within that temporary directory before merging the result into the output directory.

2. Foreach-DTO Generation

For creating aggregate files (e.g., a registration file or an index), you can use special markers in any template file. These are lines containing the following strings:

• FOREACH-DTO-START
• FOREACH-DTO-END

The block of text between these two markers will be duplicated for every DTO, with placeholders replaced accordingly. The marker lines themselves are removed - but it's suggested to use use format-specific comments such as eg. <!-- FOREACH-DTO-START --> in XML.

Example:

// FOREACH-DTO-START
builder.Services.AddDtoServer<__DtoName____DtoVersion__>();
// FOREACH-DTO-END

Available Placeholders

You can also define custom placeholders directly in your DTO files using the pattern // __placeholder_name__ value. See the Multi-Value Placeholders section below for details.

Multi-Value Placeholders with FOREACH-DTO-VALUE

You can extract multiple values for the same placeholder from a DTO file and iterate over them in templates using FOREACH-DTO-VALUE blocks.

Defining Multi-Value Placeholders

In your DTO file, specify the same placeholder multiple times on different lines:

// minor version 1
// __indexname__ bysku
// __indexname__ byname  
// __indexname__ bycategory

message Product {
    string sku = 1;
    string name = 2;
}

Using FOREACH-DTO-VALUE in Templates

Create a template with FOREACH-DTO-VALUE-START and FOREACH-DTO-VALUE-END markers:

public class ProductIndexes {
    public void RegisterIndexes() {
        // FOREACH-DTO-VALUE-START __indexname__
        RegisterIndex("__indexname__");
        // FOREACH-DTO-VALUE-END
    }
}

Generated Output:

public class ProductIndexes {
    public void RegisterIndexes() {
        RegisterIndex("bysku");
        RegisterIndex("byname");
        RegisterIndex("bycategory");
    }
}

Paired Placeholders (Format B)

You can define multiple placeholders together on the same line, and they will stay paired during iteration:

DTO File:

// __aliasname__ __aliaspattern__ byitem item_sku
// __aliasname__ __aliaspattern__ bycustomer customer_id

Template:

// FOREACH-DTO-VALUE-START __aliasname__
AddAlias("__aliasname__", "__aliaspattern__");
// FOREACH-DTO-VALUE-END

Generated Output:

AddAlias("byitem", "item_sku");
AddAlias("bycustomer", "customer_id");

Key Features:

• Placeholders appearing on the same line are automatically paired
• Both placeholders in a pair must be used consistently together
• Order matters: __name__ __pattern__ is different from __pattern__ __name__
• Mix single-value placeholders (like __dtoname__) with multi-value placeholders in FOREACH-DTO-VALUE blocks

Nesting FOREACH Blocks

You can nest FOREACH-DTO-VALUE inside FOREACH-DTO for per-DTO multi-value generation:

// FOREACH-DTO-START
public class __DtoName__Indexes {
    // FOREACH-DTO-VALUE-START __indexname__
    RegisterIndex("__dtoname__", "__indexname__");
    // FOREACH-DTO-VALUE-END
}
// FOREACH-DTO-END

Important: You cannot nest FOREACH-DTO inside FOREACH-DTO-VALUE.

Unique Value Iteration with FOREACH-UNIQUE-DTO-VALUE

When your DTO files contain duplicate values for multi-value placeholders, you can use FOREACH-UNIQUE-DTO-VALUE to iterate only over unique values, automatically filtering duplicates while preserving first-occurrence order.

Use Case: Enum Generation

A common scenario is generating enums from DTO metadata that may contain duplicates:

DTO File (order.v1.proto):

// minor version 1
// __statusvalue__ Pending
// __statusvalue__ Processing
// __statusvalue__ Pending     // duplicate
// __statusvalue__ Completed
// __statusvalue__ Processing  // duplicate
// __statusvalue__ Cancelled

Template:

public enum OrderStatus {
    // FOREACH-UNIQUE-DTO-VALUE-START __statusvalue__
    __statusvalue__,
    // FOREACH-UNIQUE-DTO-VALUE-END __statusvalue__
}

Generated Output:

public enum OrderStatus {
    Pending,
    Processing,
    Completed,
    Cancelled,
}

Key Features:

• Automatic deduplication: Duplicates are filtered automatically based on first occurrence
• Order preservation: Values appear in the order they first occur in the DTO
• Works with paired placeholders: Uniqueness is determined by the primary placeholder
• Integrates with FOREACH-DTO: Can be nested inside FOREACH-DTO blocks for per-DTO unique iteration
• Coexists with FOREACH-DTO-VALUE: Both can be used in the same template for different purposes

Deduplication with Paired Placeholders

When using paired placeholders (Format B), uniqueness is determined by the primary (first) placeholder:

DTO File:

// __aliasname__ __aliaspath__ bysku product/sku
// __aliasname__ __aliaspath__ bysku product/id    // duplicate primary
// __aliasname__ __aliaspath__ byname product/name

Template:

// FOREACH-UNIQUE-DTO-VALUE-START __aliasname__
AddUniqueAlias("__aliasname__", "__aliaspath__");
// FOREACH-UNIQUE-DTO-VALUE-END __aliasname__

Generated Output:

AddUniqueAlias("bysku", "product/sku");    // First occurrence kept
AddUniqueAlias("byname", "product/name");

The second bysku entry is filtered out because the primary placeholder (__aliasname__) is a duplicate, even though the secondary placeholder (__aliaspath__) has a different value.

Combining FOREACH-DTO-VALUE and FOREACH-UNIQUE-DTO-VALUE

You can use both in the same template when you need all values in one place and unique values in another:

public class ProductIndexes {
    // Register all indexes (including duplicates for tracking)
    public void RegisterAllIndexes() {
        // FOREACH-DTO-VALUE-START __indexname__
        RegisterIndex("__indexname__");
        // FOREACH-DTO-VALUE-END __indexname__
    }
    
    // Register only unique indexes (for schema creation)
    public void CreateUniqueIndexes() {
        // FOREACH-UNIQUE-DTO-VALUE-START __indexname__
        CreateIndex("__indexname__");
        // FOREACH-UNIQUE-DTO-VALUE-END __indexname__
    }
}

When to Use FOREACH-UNIQUE-DTO-VALUE

Use FOREACH-UNIQUE-DTO-VALUE when:

• Generating enums from potentially duplicate data
• Creating schema definitions (tables, indexes, etc.)
• Building configuration that requires unique keys
• Deduplicating import statements or dependencies

Use regular FOREACH-DTO-VALUE when:

• You need to preserve all occurrences (including duplicates)
• Counting or tracking frequency of values
• Maintaining audit trails or logs

Conditional Inclusion with IFDEF-DTO-VALUE / IFANY-DTO-VALUE

The IFDEF-DTO-VALUE and IFANY-DTO-VALUE markers allow you to conditionally include blocks of content based on whether a placeholder is defined in the DTO file. Both marker names work identically and are completely interchangeable.

Syntax

// IFDEF-DTO-VALUE-START __placeholder__
... content to include if __placeholder__ exists ...
// IFDEF-DTO-VALUE-END

Or equivalently:

// IFANY-DTO-VALUE-START __placeholder__
... content to include if __placeholder__ exists ...
// IFANY-DTO-VALUE-END

Important Notes:

• If __placeholder__ exists in the DTO (has at least one value), the content between the markers is included
• If __placeholder__ doesn't exist, the entire block (including the marker lines) is removed
• Unlike FOREACH-DTO-VALUE, this does NOT iterate - content is included at most once
• The markers themselves are removed from the output
• Placeholder substitution happens AFTER the inclusion decision
• IFDEF and IFANY are completely interchangeable - use whichever reads better in your context

Example: Optional Features

DTO with optional metadata (product.v1.proto):

// minor version 1
// __indexname__ bysku
// __cacheable__ true

message Product {
    string sku = 1;
    string name = 2;
}

DTO without optional metadata (order.v1.proto):

// minor version 1
// __indexname__ byid

message Order {
    string id = 1;
}

Template with conditional block:

public class __DtoName____DtoVersion__ {
    public string Name = "__dtoname__";
    
    // IFDEF-DTO-VALUE-START __cacheable__
    public bool IsCacheable = true;
    public string CacheKey = "__DtoName___cache";
    // IFDEF-DTO-VALUE-END
    
    public List<string> Indexes = new List<string>();
}

Generated output for Product (has cacheable):

public class ProductV1 {
    public string Name = "product";
    
    public bool IsCacheable = true;
    public string CacheKey = "Product_cache";
    
    public List<string> Indexes = new List<string>();
}

Generated output for Order (no cacheable):

public class OrderV1 {
    public string Name = "order";
    
    public List<string> Indexes = new List<string>();
}

The caching-related fields only appear in the Product class because __cacheable__ is defined in its DTO file.

Integration with FOREACH-DTO

IFDEF-DTO-VALUE works seamlessly inside FOREACH-DTO blocks, allowing each DTO to have different optional content:

// FOREACH-DTO-START
public class __DtoName____DtoVersion__Config {
    // IFDEF-DTO-VALUE-START __cacheable__
    public TimeSpan CacheDuration = TimeSpan.FromMinutes(5);
    // IFDEF-DTO-VALUE-END
    
    // IFDEF-DTO-VALUE-START __audit__
    public bool EnableAuditLog = true;
    // IFDEF-DTO-VALUE-END
}
// FOREACH-DTO-END

Each DTO will generate its own config class with only the features it declares.

When to Use IFDEF-DTO-VALUE

Use IFDEF-DTO-VALUE when:

• Different DTOs have different optional features or capabilities
• You want to keep templates DRY while supporting optional functionality
• You need conditional compilation patterns (e.g., feature flags)
• Some DTOs require extra configuration or initialization

This is particularly useful for:

• Optional caching strategies
• Conditional validation rules
• Feature-specific code generation
• DTO-specific optimizations

Conditional Inclusion with IFNDEF-DTO-VALUE / IFNO-DTO-VALUE

The IFNDEF-DTO-VALUE and IFNO-DTO-VALUE markers provide the opposite behavior of IFDEF/IFANY - they include content only when a placeholder is NOT defined in the DTO file. Both marker names work identically and are completely interchangeable.

Syntax

// IFNDEF-DTO-VALUE-START __placeholder__
... content to include if __placeholder__ does NOT exist ...
// IFNDEF-DTO-VALUE-END

Or equivalently:

// IFNO-DTO-VALUE-START __placeholder__
... content to include if __placeholder__ does NOT exist ...
// IFNO-DTO-VALUE-END

Important Notes:

• If __placeholder__ does NOT exist in the DTO, the content between the markers is included
• If __placeholder__ exists (has at least one value), the entire block is removed
• This is the logical opposite of IFDEF/IFANY
• The markers themselves are removed from the output
• IFNDEF and IFNO are completely interchangeable

Example: Default Values

DTO with custom cache (product.v1.proto):

// minor version 1
// __custom_cache__ redis
// __custom_timeout__ 3600

DTO without custom cache (order.v1.proto):

// minor version 1

Template:

// FOREACH-DTO-START
public class __DtoName____DtoVersion__Config {
    public string Name = "__dtoname__";
    
    // IFNDEF-DTO-VALUE-START __custom_cache__
    public string CacheStrategy = "default";
    // IFNDEF-DTO-VALUE-END
    
    // IFNO-DTO-VALUE-START __custom_timeout__
    public int TimeoutSeconds = 30;
    // IFNO-DTO-VALUE-END
}
// FOREACH-DTO-END

Generated output for Product (has custom values):

public class ProductV1Config {
    public string Name = "product";
}

Generated output for Order (no custom values):

public class OrderV1Config {
    public string Name = "order";
    
    public string CacheStrategy = "default";
    
    public int TimeoutSeconds = 30;
}

The default values only appear when the custom placeholders are not defined.

Combining IFDEF and IFNDEF

You can mix both types of conditional markers in the same template:

// FOREACH-DTO-START
public class __DtoName____DtoVersion__ {
    // IFDEF-DTO-VALUE-START __custom_validation__
    public IValidator Validator = new CustomValidator();
    // IFDEF-DTO-VALUE-END
    
    // IFNDEF-DTO-VALUE-START __custom_validation__
    public IValidator Validator = new DefaultValidator();
    // IFNDEF-DTO-VALUE-END
}
// FOREACH-DTO-END

When to Use IFNDEF-DTO-VALUE

Use IFNDEF-DTO-VALUE when:

• You want to provide default values for DTOs that don't customize them
• Creating fallback behavior when optional features aren't specified
• Generating boilerplate only for DTOs without custom implementations
• Implementing "opt-out" patterns (default is on, custom disables it)

This is particularly useful for:

• Default configuration values
• Standard initialization code
• Fallback implementations
• Base-case code generation

Gitignore Support

By default, dtox respects .gitignore files when processing templates. This helps prevent binary files, build artifacts, and other unwanted files from being included in the generated output.

Default Behavior (Gitignore Enabled):

• Automatically skips files and directories specified in .gitignore
• Prevents binary files from causing UTF-8 encoding errors
• Follows standard git ignore rules (.gitignore, .git/info/exclude, global gitignore)
• Logs when gitignore rules are being applied

Disabling Gitignore: Use the --no-gitignore flag to include all files, even those normally ignored:

dtox --input-dir ./templates --output-dir ./output --no-gitignore

Common Use Cases:

• Enabled (default): Normal template processing where you want to exclude build artifacts, IDE files, etc.
• Disabled (--no-gitignore): When you need to include specific files that are gitignored, or when working with repositories that don't use git

Using just (Task Runner)

This project includes a justfile to simplify common development tasks. just is a command runner similar to make or invoke.

First, install just.

Then you can use the following commands from the project directory:

• just build: Compiles the project.
• just run -- --input-dir <path>: Runs the generator. Pass arguments after --.
• just completion-bash: Generates the Bash completion script (dtox-completion.bash).
• just completion-pwsh: Generates the PowerShell completion script (_dtox.ps1).

Running just with no arguments will list all available commands.

Shell Completion

This tool can generate shell completion scripts for various shells like Bash and PowerShell. You can generate them manually or use the just recipes above.

Bash

1. Generate the completion script: Run your compiled application with the --generate-completion bash flag and redirect the output to a file.

   # Assuming your executable is in the default debug location
   ./target/debug/dtox --generate-completion bash > dtox-completion.bash
   

2. Load the script for the current session: Source the generated file to enable completion in your current terminal session.

   source dtox-completion.bash
   

3. Load the script permanently: To make completion available in all new shell sessions, you can either copy the script to a standard completion directory or source it from your shell's profile file (~/.bashrc or ~/.bash_profile).

   Option A: Copy to completion directory (recommended on Linux):

   # The exact path may vary based on your distribution
   sudo cp dtox-completion.bash /etc/bash_completion.d/dtox
   

   Option B: Source from .bashrc: Add the following line to your ~/.bashrc file, replacing /path/to/ with the actual path to the script.

   echo 'source /path/to/dtox-completion.bash' >> ~/.bashrc
   

   You will need to restart your shell for the changes to take effect.

PowerShell (pwsh)

1. Check your execution policy: PowerShell requires scripts to be signed or the execution policy to be changed. You can check your current policy with Get-ExecutionPolicy. If it's Restricted, you may need to change it.

   # Run this command in a PowerShell terminal with administrator privileges
   Set-ExecutionPolicy RemoteSigned -Scope CurrentUser
   

2. Generate the completion script:

   # Assuming your executable is in the default debug location
   .\target\debug\dtox.exe --generate-completion powershell > _dtox.ps1
   

3. Load the script permanently: To load the script every time you open PowerShell, add it to your profile. You can find your profile path by checking the $PROFILE variable.

   # Open your profile in Notepad (it will be created if it doesn't exist)
   notepad $PROFILE
   

   Add the following line to the profile file, replacing C:\path\to\ with the actual path to the script.

   . C:\path\to\_dtox.ps1
   

   Save the file and restart your PowerShell session. You can now use tab-completion for the dtox command and its arguments.