Tested with Travis CI

1. Overview
2. Module Description - What the module does and why it is useful
3. Setup - The basics of getting started with carbon
   • What carbon affects
   • Beginning with carbon
4. Usage - Configuration options and additional functionality
   • Classes and Defined Types
     • Class: carbon
     • Defined Type: carbon::aggregation_rule
     • Defined Type: carbon::aggregator
     • Defined Type: carbon::blacklist
     • Defined Type: carbon::cache
     • Defined Type: carbon::relay
     • Defined Type: carbon::relay_rule
     • Defined Type: carbon::rewrite_rule
     • Defined Type: carbon::storage_aggregation
     • Defined Type: carbon::storage_schema
     • Defined Type: carbon::whitelist
   • Examples
5. Reference - An under-the-hood peek at what the module is doing and how
6. Limitations - OS compatibility, etc.
7. Development - Guide for contributing to the module

This module manages the Carbon daemons.

This module installs the Carbon packages, configures the carbon, relay, and aggregator daemons and runs them. It currently leverages systemd to support running multiple instances of each. It also manages the various rules and lists supported by each.

If running on Puppet 3.x you will need to have the future parser enabled. On RHEL/CentOS platforms you will need to enable the EPEL repository first.

• The package(s) providing the Carbon software.
• The various /etc/carbon/*.conf configuration files.
• The systemd unit file for each daemon instance.

include ::carbon

Parameters within carbon:

A hash of carbon::aggregation_rule instances to create.

A hash of carbon::aggregator instances to create.

A hash of carbon::blacklist instances to create.

A hash of carbon::cache instances to create.

The base configuration directory, defaults to /etc/carbon.

The directory where whisper files are written, defaults to ${storage_dir}/whisper.

The log directory, defaults to /var/log/carbon.

The name of the package to install that provides the Carbon software.

The directory where PID files are written, defaults to /var/run.

A hash of carbon::relay_rule instances to create.

A hash of carbon::relay instances to create.

A hash of carbon::rewrite_rule instances to create.

The base storage dіrectory, defaults to /var/lib/carbon.

A hash of carbon::storage_aggregation instances to create.

A hash of carbon::storage_schema instances to create.

The user to run as, defaults to carbon.

A hash of carbon::whitelist instances to create.

The directory used for whitelisting, defaults to /var/lib/carbon/lists.

Parameters within carbon::aggregation_rule:

The name of the rule.

How frequently to aggregate.

Input pattern for matching incoming metrics.

One of avg or sum.

The template for the outgoing aggregated metrics.

Used to place the rule in the file, defaults to '10'.

Parameters within carbon::aggregator:

The name of the aggregator instance, should be a single letter a-z.

The running state of the instance, should be stopped or running.

Boolean whether to enable the instance.

Maps to the LINE_RECEIVER_INTERFACE setting.

Maps to the LINE_RECEIVER_PORT setting.

Maps to the PICKLE_RECEIVER_INTERFACE setting.

Maps to the PICKLE_RECEIVER_PORT setting.

Maps to the LOG_LISTENER_CONNECTIONS setting.

Maps to the FORWARD_ALL setting.

An array of hashes for each destination instance. Hash must contain host and port keys and optionally an instance key. Builds the DESTINATIONS setting.

Maps to the REPLICATION_FACTOR setting.

Maps to the MAX_QUEUE_SIZE setting.

Maps to the USE_FLOW_CONTROL setting.

Maps to the MAX_DATAPOINTS_PER_MESSAGE setting.

Maps to the MAX_AGGREGATION_INTERVALS setting.

Maps to the WRITE_BACK_FREQUENCY setting.

Maps to the USE_WHITELIST setting.

Maps to the CARBON_METRIC_PREFIX setting.

Maps to the CARBON_METRIC_INTERVAL setting.

Parameters within carbon::blacklist:

The name of the blacklist entry.

The regular expression for the blacklist pattern.

Where to place the blacklist in the file, defaults to '10'.

Parameters within carbon::cache:

The name of the cache instance, should be a single letter a-z.

The running state of the instance, should be stopped or running.

Boolean whether to enable the instance.

Maps to the STORAGE_DIR setting. Defaults to /var/lib/carbon.

Maps to the LOCAL_DATA_DIR setting. Defaults to ${storage_dir}/whisper.

Maps to the WHITELISTS_DIR setting. Defaults to ${storage_dir}/lists.

Maps to the CONF_DIR setting. Defaults to /etc/carbon.

Maps to the LOG_DIR setting. Defaults to /var/log/carbon.

Maps to the PID_DIR setting. Defaults to /var/run.

Maps to the ENABLE_LOGROTATION setting.

Maps to the USER setting. Defaults to carbon.

Maps to the MAX_CACHE_SIZE setting.

Maps to the MAX_UPDATES_PER_SECOND setting.

Maps to the MAX_UPDATES_PER_SECOND_ON_SHUTDOWN setting.

Maps to the MAX_CREATES_PER_MINUTE setting.

Maps to the LINE_RECEIVER_INTERFACE setting.

Maps to the LINE_RECEIVER_PORT setting.

Maps to the LINE_RECEIVER_BACKLOG setting.

Maps to the ENABLE_UDP_LISTENER setting.

Maps to the UDP_RECEIVER_INTERFACE setting.

Maps to the UDP_RECEIVER_PORT setting.

Maps to the PICKLE_RECEIVER_INTERFACE setting.

Maps to the PICKLE_RECEIVER_PORT setting.

Maps to the PICKLE_RECEIVER_BACKLOG setting.

Maps to the LOG_LISTENER_CONNECTIONS setting.

Maps to the USE_INSECURE_UNPICKLER setting.

Maps to the CACHE_QUERY_INTERFACE setting.

Maps to the CACHE_QUERY_PORT setting.

Maps to the CACHE_QUERY_BACKLOG setting.

Maps to the USE_FLOW_CONTROL setting.

Maps to the LOG_UPDATES setting.

Maps to the LOG_CACHE_HITS setting.

Maps to the LOG_CACHE_QUEUE_SORTS setting.

Maps to the CACHE_WRITE_STRATEGY setting.

Maps to the WHISPER_AUTOFLUSH setting.

Maps to the WHISPER_SPARSE_CREATE setting.

Maps to the WHISPER_FALLOCATE_CREATE setting.

Maps to the WHISPER_LOCK_WRITES setting.

Maps to the USE_WHITELIST setting.

Maps to the CARBON_METRIC_PREFIX setting.

Maps to the CARBON_METRIC_INTERVAL setting.

Maps to the ENABLE_AMQP setting.

Maps to the AMQP_VERBOSE setting.

Maps to the AMQP_HOST setting.

Maps to the AMQP_PORT setting.

Maps to the AMQP_VHOST setting.

Maps to the AMQP_USER setting.

Maps to the AMQP_PASSWORD setting.

Maps to the AMQP_EXCHANGE setting.

Maps to the AMQP_METRIC_NAME_IN_BODY setting.

Maps to the ENABLE_MANHOLE setting.

Maps to the MANHOLE_INTERFACE setting.

Maps to the MANHOLE_PORT setting.

Maps to the MANHOLE_USER setting.

Maps to the MANHOLE_PUBLIC_KEY setting.

An array of AMQP bind patterns. Builds the BIND_PATTERNS setting.

Parameters within carbon::relay:

The name of the relay instance, should be a single letter a-z.

The running state of the instance, should be stopped or running.

Boolean whether to enable the instance.

Maps to the LINE_RECEIVER_INTERFACE setting.

Maps to the LINE_RECEIVER_PORT setting.

Maps to the PICKLE_RECEIVER_INTERFACE setting.

Maps to the PICKLE_RECEIVER_PORT setting.

Maps to the LOG_LISTENER_CONNECTIONS setting.

Maps to the RELAY_METHOD setting. One of rules, consistent-hashing, or aggregated-consistent-hashing.

Maps to the REPLICATION_FACTOR setting.

Maps to the DIVERSE_REPLICAS setting.

An array of hashes for each destination instance. Hash must contain host and port keys and optionally an instance key. Builds the DESTINATIONS setting.

Maps to the MAX_DATAPOINTS_PER_MESSAGE setting.

Maps to the MAX_QUEUE_SIZE setting.

Maps to the QUEUE_LOW_WATERMARK_PCT setting.

Maps to the USE_FLOW_CONTROL setting.

Maps to the USE_WHITELIST setting.

Maps to the CARBON_METRIC_PREFIX setting.

Maps to the CARBON_METRIC_INTERVAL setting.

Parameters within carbon::relay_rule:

The name of the rule.

An array of hashes for each destination instance. Hash must contain host and port keys and optionally an instance key.

Whether to drop through to further rules.

Marks the default relay rule.

The regular expression for matching metrics.

Used to place the rule in the file, defaults to '10'.

Parameters within carbon::rewrite_rule:

The name of the rule.

The regular expression for matching metrics.

One of pre or post.

The replacement pattern to apply.

Used to place the rule in the file, defaults to '10'.

Parameters within carbon::storage_aggregation:

The name of the rule.

One of average, sum, last, max or min.

The regular expression for matching metrics.

Represents the ratio of datapoints needed in order to aggregate accurately.

Used to place the rule in the file, defaults to '10'.

Parameters within carbon::storage_schema:

The name of the rule.

The regular expression for matching metrics.

An array of data retention policies.

Used to place the rule in the file, defaults to '10'.

Parameters within carbon::whitelist:

The name of the whitelist entry.

The regular expression for the whitelist pattern.

Where to place the whitelist in the file, defaults to '10'.

Install and create a sole cache instance (package defaults):

include ::carbon

Extend the above to add a second cache instance with a relay instance in front of both balancing metrics between them.

class { '::carbon':
  relays => {},
}

::carbon::cache { 'b':
  ensure               => running,
  enable               => true,
  line_receiver_port   => 2103,
  pickle_receiver_port => 2104,
  cache_query_port     => 7102,
}

::carbon::relay { 'a':
  ensure                     => running,
  enable                     => true,
  line_receiver_interface    => '0.0.0.0',
  line_receiver_port         => 2013,
  pickle_receiver_interface  => '0.0.0.0',
  pickle_receiver_port       => 2014,
  log_listener_connections   => true,
  relay_method               => 'consistent-hashing',
  replication_factor         => 1,
  destinations               => [
    {
      'host'     => '127.0.0.1',
      'port'     => 2004,
      'instance' => 'a',
    },
    {
      'host'     => '127.0.0.1',
      'port'     => 2104,
      'instance' => 'b',
    },
  ],
  max_datapoints_per_message => 500,
  max_queue_size             => 10000,
  queue_low_watermark_pct    => 0.8,
  use_flow_control           => true,
}

• carbon: Main class for managing the Carbon daemons.

• carbon::install: Handles Carbon installation.
• carbon::config: Handles Carbon configuration.
• carbon::params: Different configuration data for different systems.
• carbon::service: Handles stopping the default services.

• carbon::aggregation_rule: Handles aggregation rules.
• carbon::aggregator: Handles creating aggregator instances.
• carbon::blacklist: Handles blacklist rules.
• carbon::cache: Handles creating cache instances.
• carbon::relay: Handles creating relay instances.
• carbon::relay_rule: Handles relay rules.
• carbon::rewrite_rule: Handles rewrite rules.
• carbon::storage_aggregation: Handles storage aggregation rules.
• carbon::storage_schema: Handles storage schema rules.
• carbon::whitelist: Handles whitelist rules.

This module leverages systemd to create multiple instances of the three daemons.

This module has been built on and tested against Puppet 3.0 and higher.

The module has been tested on:

• RedHat/CentOS Enterprise Linux 7

Testing on other platforms has been light and cannot be guaranteed.

Please log issues or pull requests at github.