add metrics otelcol_exporter_queue_capacity #5475

fatsheep9146 · 2022-06-04T15:15:04Z

Signed-off-by: Ziqi Zhao [email protected]

Description:
Fix #4902, add metric otelcol_exporter_queue_capacity

Link to tracking Issue:
#4902

Testing:

Documentation:

github-actions · 2022-06-19T03:43:10Z

This PR was marked stale due to lack of activity. It will be closed in 14 days.

fatsheep9146 · 2022-06-19T12:24:02Z

@bogdandrutu Could you help remove this stale label or review this pr? I saw that @jpkrohling is on leave now.

codeboten

The change looks good, please add a test to TestQueuedRetry_QueueMetricsReported to ensure the metric is reported

codeboten

Please also add an entry to the changelog noting the new metric

codecov · 2022-06-26T12:30:36Z

Codecov Report

Merging #5475 (bc61d26) into main (0509360) will decrease coverage by 0.05%.
The diff coverage is 75.00%.

@@            Coverage Diff             @@
##             main    #5475      +/-   ##
==========================================
- Coverage   91.53%   91.48%   -0.06%     
==========================================
  Files         192      192              
  Lines       11398    11410      +12     
==========================================
+ Hits        10433    10438       +5     
- Misses        770      775       +5     
- Partials      195      197       +2

Impacted Files	Coverage Δ
exporter/exporterhelper/queued_retry_inmemory.go	`91.89% <50.00%> (-3.70%)`	⬇️
exporter/exporterhelper/obsreport.go	`100.00% <100.00%> (ø)`
pdata/internal/common.go	`91.85% <0.00%> (-0.75%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0509360...bc61d26. Read the comment docs.

fatsheep9146 · 2022-06-26T12:49:51Z

The change looks good, please add a test to TestQueuedRetry_QueueMetricsReported to ensure the metric is reported

Done

fatsheep9146 · 2022-06-26T12:50:03Z

Please also add an entry to the changelog noting the new metric

@codeboten Done

fatsheep9146 · 2022-06-26T23:18:42Z

@codeboten It's really weird, the report said the TestQueuedRetry_QueueMetricsReported failed, but I run the test by myself using exactly same branch and in mac and linux, they both passed. Do you know what kind of reason may cause this, or what is the difference between the CI and local development?

TylerHelmuth

Left a small nit but otherwise LGTM.

TylerHelmuth · 2022-06-28T22:06:31Z

exporter/exporterhelper/obsreport.go

@@ -55,6 +56,12 @@ func newInstruments(registry *metric.Registry) *instruments {
 		metric.WithLabelKeys(obsmetrics.ExporterKey),
 		metric.WithUnit(metricdata.UnitDimensionless))

+	insts.queueCapacity, _ = registry.AddInt64DerivedGauge(
+		obsmetrics.ExporterKey+"/queue_capacity",
+		metric.WithDescription("Current capacity of the retry queue (in batches)"),


I feel like "Current" implies the capacity can change at some point, but until the collector is restarted it is a fixed size right? Maybe something like "Fixed capacity..." is clearer?

You are right, I will fix this

TylerHelmuth · 2022-06-28T22:11:26Z

the report said the TestQueuedRetry_QueueMetricsReported failed, but I run the test by myself using exactly same branch and in mac and linux, they both passed. Do you know what kind of reason may cause this, or what is the difference between the CI and local development?

@fatsheep9146 the latest runs show TestQueuedRetry_QueueMetricsReported passing. Do you see the same?

fatsheep9146 · 2022-06-29T23:20:18Z

the report said the TestQueuedRetry_QueueMetricsReported failed, but I run the test by myself using exactly same branch and in mac and linux, they both passed. Do you know what kind of reason may cause this, or what is the difference between the CI and local development?

@fatsheep9146 the latest runs show TestQueuedRetry_QueueMetricsReported passing. Do you see the same?

No I didn't see the same result. The result I saw is as following

=== RUN   TestQueuedRetry_QueueMetricsReported
    queued_retry_test.go:515: 
        	Error Trace:	queued_retry_test.go:515
        	            				queued_retry_test.go:345
        	Error:      	could not find metric exporter/queue_capacity with tags [{{exporter} test}] reported
        	Test:       	TestQueuedRetry_QueueMetricsReported
--- FAIL: TestQueuedRetry_QueueMetricsReported (0.00s)

Yet I run test TestQueuedRetry_QueueMetricsReported in local, the test case passed. I'm really confused.

TylerHelmuth · 2022-06-30T15:21:36Z

You are correct, I see the failure you are seeing. I'm not sure why it happens.

fatsheep9146 · 2022-07-09T14:24:13Z

You are correct, I see the failure you are seeing. I'm not sure why it happens.

@TylerHelmuth
I found the reason, I didn't add metric to queued_retry_experimental, so when run go test with tag enable_unstable, the test case will fail. I already fix it.

fatsheep9146 · 2022-07-09T14:24:54Z

@codeboten @jpkrohling ping for review this pr again, thanks :-)

TylerHelmuth · 2022-07-11T15:47:00Z

@fatsheep9146 @codeboten does the docs/monitoring.md need updated as part of this new metric add? That doc seems out of date in general.

Signed-off-by: Ziqi Zhao <[email protected]>

fatsheep9146 · 2022-07-18T23:24:11Z

@TylerHelmuth @jpkrohling The monitoring doc are also updated. Please help review this again =D

TylerHelmuth

2 small nits in the new documentation, otherwise LGTM

docs/monitoring.md

Co-authored-by: Tyler Helmuth <[email protected]>

fatsheep9146 requested review from a team and tigrannajaryan June 4, 2022 15:15

fatsheep9146 mentioned this pull request Jun 4, 2022

Report exporter queue capacity as a metric #4902

Closed

github-actions bot added the Stale label Jun 19, 2022

github-actions bot removed the Stale label Jun 20, 2022

fatsheep9146 mentioned this pull request Jun 21, 2022

REQUEST: New membership for @fatsheep9146 open-telemetry/community#1089

Closed

6 tasks

codeboten reviewed Jun 24, 2022

View reviewed changes

fatsheep9146 force-pushed the add-queue-capacity-metric branch from d6aaaf0 to c2b5f0f Compare June 26, 2022 12:28

TylerHelmuth approved these changes Jun 28, 2022

View reviewed changes

fatsheep9146 force-pushed the add-queue-capacity-metric branch from aec2071 to 9ff9377 Compare June 29, 2022 22:51

fatsheep9146 force-pushed the add-queue-capacity-metric branch from b00150a to 30b28de Compare July 9, 2022 13:43

fatsheep9146 added 6 commits July 17, 2022 23:50

add metrics otelcol_exporter_queue_capacity

9f60666

Signed-off-by: Ziqi Zhao <[email protected]>

add unit test for new metric

06254ad

Signed-off-by: Ziqi Zhao <[email protected]>

add entry to changelog

2519551

Signed-off-by: Ziqi Zhao <[email protected]>

fix unittest

d49b152

Signed-off-by: Ziqi Zhao <[email protected]>

fix for reviews

c6da553

Signed-off-by: Ziqi Zhao <[email protected]>

add queue capacity metric for experimental query setting

73fe155

Signed-off-by: Ziqi Zhao <[email protected]>

update monitoring doc

e8353e1

Signed-off-by: Ziqi Zhao <[email protected]>

fatsheep9146 force-pushed the add-queue-capacity-metric branch from 30b28de to e8353e1 Compare July 18, 2022 23:22

jpkrohling approved these changes Jul 19, 2022

View reviewed changes

TylerHelmuth approved these changes Jul 19, 2022

View reviewed changes

docs/monitoring.md Outdated Show resolved Hide resolved

docs/monitoring.md Outdated Show resolved Hide resolved

fatsheep9146 and others added 3 commits July 19, 2022 22:51

Update docs/monitoring.md

8d8c14a

Co-authored-by: Tyler Helmuth <[email protected]>

Update docs/monitoring.md

5765c91

Co-authored-by: Tyler Helmuth <[email protected]>

Update CHANGELOG.md

bc61d26

codeboten approved these changes Jul 19, 2022

View reviewed changes

codeboten merged commit 389c047 into open-telemetry:main Jul 19, 2022

mx-psi mentioned this pull request Sep 14, 2022

[docs] Update release instructions #6068

Merged

This was referenced Sep 28, 2022

[receiver/otlp] Rewrite logs related to server startup #6174

Merged

[pdata] Update deprecation comments #6206

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add metrics otelcol_exporter_queue_capacity #5475

add metrics otelcol_exporter_queue_capacity #5475

fatsheep9146 commented Jun 4, 2022

github-actions bot commented Jun 19, 2022

fatsheep9146 commented Jun 19, 2022

codeboten left a comment

codeboten left a comment

codecov bot commented Jun 26, 2022 •

edited

Loading

fatsheep9146 commented Jun 26, 2022

fatsheep9146 commented Jun 26, 2022

fatsheep9146 commented Jun 26, 2022

TylerHelmuth left a comment

TylerHelmuth Jun 28, 2022

fatsheep9146 Jun 29, 2022

TylerHelmuth commented Jun 28, 2022

fatsheep9146 commented Jun 29, 2022

TylerHelmuth commented Jun 30, 2022

fatsheep9146 commented Jul 9, 2022

fatsheep9146 commented Jul 9, 2022

TylerHelmuth commented Jul 11, 2022

fatsheep9146 commented Jul 18, 2022

TylerHelmuth left a comment

add metrics otelcol_exporter_queue_capacity #5475

add metrics otelcol_exporter_queue_capacity #5475

Conversation

fatsheep9146 commented Jun 4, 2022

github-actions bot commented Jun 19, 2022

fatsheep9146 commented Jun 19, 2022

codeboten left a comment

Choose a reason for hiding this comment

codeboten left a comment

Choose a reason for hiding this comment

codecov bot commented Jun 26, 2022 • edited Loading

Codecov Report

fatsheep9146 commented Jun 26, 2022

fatsheep9146 commented Jun 26, 2022

fatsheep9146 commented Jun 26, 2022

TylerHelmuth left a comment

Choose a reason for hiding this comment

TylerHelmuth Jun 28, 2022

Choose a reason for hiding this comment

fatsheep9146 Jun 29, 2022

Choose a reason for hiding this comment

TylerHelmuth commented Jun 28, 2022

fatsheep9146 commented Jun 29, 2022

TylerHelmuth commented Jun 30, 2022

fatsheep9146 commented Jul 9, 2022

fatsheep9146 commented Jul 9, 2022

TylerHelmuth commented Jul 11, 2022

fatsheep9146 commented Jul 18, 2022

TylerHelmuth left a comment

Choose a reason for hiding this comment

codecov bot commented Jun 26, 2022 •

edited

Loading