Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add metrics otelcol_exporter_queue_capacity #5475

Merged
merged 10 commits into from
Jul 19, 2022
Merged

add metrics otelcol_exporter_queue_capacity #5475

Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
9f60666
add metrics otelcol_exporter_queue_capacity
fatsheep9146 Jun 4, 2022
06254ad
add unit test for new metric
fatsheep9146 Jun 26, 2022
2519551
add entry to changelog
fatsheep9146 Jun 26, 2022
d49b152
fix unittest
fatsheep9146 Jun 26, 2022
c6da553
fix for reviews
fatsheep9146 Jun 29, 2022
73fe155
add queue capacity metric for experimental query setting
fatsheep9146 Jul 9, 2022
e8353e1
update monitoring doc
fatsheep9146 Jul 18, 2022
8d8c14a
Update docs/monitoring.md
fatsheep9146 Jul 19, 2022
5765c91
Update docs/monitoring.md
fatsheep9146 Jul 19, 2022
bc61d26
Update CHANGELOG.md
codeboten Jul 19, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update monitoring doc 
Signed-off-by: Ziqi Zhao <[email protected]>
  • Loading branch information
@fatsheep9146
fatsheep9146 committed Jul 18, 2022
commit e8353e1bba05e6eacb494dd259c395137fb2439b
6 changes: 4 additions & 2 deletions docs/monitoring.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,11 @@ Most exporters offer a [queue/retry mechanism](../exporter/exporterhelper/README
that is recommended as the retry mechanism for the Collector and as such should
be used in any production deployment.

**TODO:** Add metric to monitor queue length.
The `otelcol_exporter_queue_capacity` indicates the capacity of the retry queue (in batches). The `otelcol_exporter_queue_size` indicates the current size of retry queue. So you can check those two metrics to see if queue size is enough for you load.
fatsheep9146 marked this conversation as resolved.
Show resolved Hide resolved

Currently, the queue/retry mechanism only supports logging for monitoring. Check
The `otelcol_exporter_enqueue_failed_spans`, `otelcol_exporter_enqueue_failed_metric_points` and `otelcol_exporter_enqueue_failed_log_records` indicate the number of span/metric points/log records failed to be added to the sending queue. This maybe cause by the queue is full of unsettled elements so you should decrease your sending rate or horizontally scale your collector.
fatsheep9146 marked this conversation as resolved.
Show resolved Hide resolved

The queue/retry mechanism also supports logging for monitoring. Check
the logs for messages like `"Dropping data because sending_queue is full"`.

### Receive Failures
Expand Down