Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set grpc logger to collector #4501

Merged
merged 11 commits into from
Dec 13, 2021
Merged

Set grpc logger to collector #4501

merged 11 commits into from
Dec 13, 2021

Conversation

pavankrish123
Copy link
Contributor

@pavankrish123 pavankrish123 commented Dec 1, 2021

Description:

Adding a feature - This feature sets the grpc logger to one derived from collector logger configuration and enables users to control grpc logs.

  • In the case of collector default configuration, gRPC logs minimum level is set to WARN (to reduce the log traffic) but shared rest of collector's logger configuration.
$ ./bin/cmd-otelcol --config ./bin/config.yaml
2021-11-30T16:47:38.293-0800	info	service/collector.go:201	Applying configuration...
2021-11-30T16:47:38.294-0800	info	builder/exporters_builder.go:254	Exporter was built.	{"kind": "exporter", "name": "otlp"}
2021-11-30T16:47:38.294-0800	info	builder/exporters_builder.go:254	Exporter was built.	{"kind": "exporter", "name": "logging"}
2021-11-30T16:47:38.295-0800	info	builder/pipelines_builder.go:222	Pipeline was built.	{"name": "pipeline", "name": "metrics"}
2021-11-30T16:47:38.295-0800	info	builder/pipelines_builder.go:222	Pipeline was built.	{"name": "pipeline", "name": "traces"}
2021-11-30T16:47:38.295-0800	info	builder/receivers_builder.go:224	Receiver was built.	{"kind": "receiver", "name": "otlp", "datatype": "traces"}
2021-11-30T16:47:38.295-0800	info	builder/receivers_builder.go:224	Receiver was built.	{"kind": "receiver", "name": "otlp", "datatype": "metrics"}
2021-11-30T16:47:38.295-0800	info	service/service.go:86	Starting extensions...
2021-11-30T16:47:38.295-0800	info	service/service.go:91	Starting exporters...
2021-11-30T16:47:38.295-0800	info	builder/exporters_builder.go:40	Exporter is starting...	{"kind": "exporter", "name": "logging"}
2021-11-30T16:47:38.295-0800	info	builder/exporters_builder.go:48	Exporter started.	{"kind": "exporter", "name": "logging"}
2021-11-30T16:47:38.295-0800	info	builder/exporters_builder.go:40	Exporter is starting...	{"kind": "exporter", "name": "otlp"}
2021-11-30T16:47:38.295-0800	info	builder/exporters_builder.go:48	Exporter started.	{"kind": "exporter", "name": "otlp"}
2021-11-30T16:47:38.295-0800	info	service/service.go:96	Starting processors...
2021-11-30T16:47:38.295-0800	info	builder/pipelines_builder.go:54	Pipeline is starting...	{"name": "pipeline", "name": "metrics"}
2021-11-30T16:47:38.295-0800	info	builder/pipelines_builder.go:65	Pipeline is started.	{"name": "pipeline", "name": "metrics"}
2021-11-30T16:47:38.295-0800	info	builder/pipelines_builder.go:54	Pipeline is starting...	{"name": "pipeline", "name": "traces"}
2021-11-30T16:47:38.295-0800	info	builder/pipelines_builder.go:65	Pipeline is started.	{"name": "pipeline", "name": "traces"}
2021-11-30T16:47:38.295-0800	info	service/service.go:101	Starting receivers...
2021-11-30T16:47:38.295-0800	info	builder/receivers_builder.go:68	Receiver is starting...	{"kind": "receiver", "name": "otlp"}
2021-11-30T16:47:38.295-0800	info	otlpreceiver/otlp.go:68	Starting GRPC server on endpoint 0.0.0.0:4317	{"kind": "receiver", "name": "otlp"}
2021-11-30T16:47:38.302-0800	info	builder/receivers_builder.go:73	Receiver started.	{"kind": "receiver", "name": "otlp"}
2021-11-30T16:47:38.302-0800	info	service/telemetry.go:92	Setting up own telemetry...
2021-11-30T16:47:38.310-0800	info	service/telemetry.go:116	Serving Prometheus metrics	{"address": ":8888", "level": "basic", "service.instance.id": "a8caf36a-00df-407e-b356-23569252e525", "service.version": "latest"}
2021-11-30T16:47:38.310-0800	info	service/collector.go:250	Starting cmd-otelcol...	{"Version": "0.40.0-dev", "NumCPU": 12}
2021-11-30T16:47:38.310-0800	info	service/collector.go:138	Everything is ready. Begin running and processing data.
2021-11-30T16:47:38.384-0800	warn	zapgrpc/zapgrpc.go:191	[core] grpc: addrConn.createTransport failed to connect to {otelcol:4317 otelcol:4317 <nil> <nil> 0 <nil>}. Err: connection error: desc = "transport: Error while dialing dial tcp: lookup otelcol: no such host"
2021-11-30T16:47:42.912-0800	INFO	loggingexporter/logging_exporter.go:40	TracesExporter	{"#spans": 200}
  • for all other levels - gRPC logs min log level takes whatever is set for collector log sharing rest of collector's logger configuration. for example
service:
  telemetry:
    logs:
      level: "DEBUG"
      encoding: "console"
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging, otlp]

in non development made

$ ./bin/cmd-otelcol --config ./bin/config.yaml
2021-11-30T17:35:09.442-0800	info	builder/exporters_builder.go:40	Exporter is starting...	{"kind": "exporter", "name": "otlp"}
2021-11-30T17:35:09.442-0800	info	zapgrpc/zapgrpc.go:174	[core] original dial target is: "otelcol:4317"
2021-11-30T17:35:09.443-0800	info	zapgrpc/zapgrpc.go:174	[core] parsed dial target is: {Scheme:otelcol Authority: Endpoint:4317 URL:{Scheme:otelcol Opaque:4317 User: Host: Path: RawPath: ForceQuery:false RawQuery: Fragment: RawFragment:}}
2021-11-30T17:35:09.443-0800	info	zapgrpc/zapgrpc.go:174	[core] fallback to scheme "passthrough"
2021-11-30T17:35:09.443-0800	info	zapgrpc/zapgrpc.go:174	[core] parsed dial target is: {Scheme:passthrough Authority: Endpoint:otelcol:4317 URL:{Scheme:passthrough Opaque: User: Host: Path:/otelcol:4317 RawPath: ForceQuery:false RawQuery: Fragment: RawFragment:}}
2021-11-30T17:35:09.443-0800	info	zapgrpc/zapgrpc.go:174	[core] Channel authority set to "otelcol:4317"
2021-11-30T17:35:09.443-0800	info	zapgrpc/zapgrpc.go:174	[core] ccResolverWrapper: sending update to cc: {[{otelcol:4317  <nil> <nil> 0 <nil>}] <nil> <nil>}
2021-11-30T17:35:09.443-0800	info	zapgrpc/zapgrpc.go:174	[core] ClientConn switching balancer to "pick_first"
2021-11-30T17:35:09.443-0800	info	zapgrpc/zapgrpc.go:174	[core] Channel switches to new LB policy "pick_first"
2021-11-30T17:35:09.443-0800	info	builder/exporters_builder.go:48	Exporter started.	{"kind": "exporter", "name": "otlp"}
2021-11-30T17:35:09.443-0800	info	zapgrpc/zapgrpc.go:174	[core] Subchannel Connectivity change to CONNECTING
2021-11-30T17:35:09.443-0800	info	builder/exporters_builder.go:40	Exporter is starting...	{"kind": "exporter", "name": "logging"}
2021-11-30T17:35:09.443-0800	info	builder/exporters_builder.go:48	Exporter started.	{"kind": "exporter", "name": "logging"}
2021-11-30T17:35:09.443-0800	info	service/service.go:96	Starting processors...
2021-11-30T17:35:09.443-0800	info	builder/pipelines_builder.go:54	Pipeline is starting...	{"name": "pipeline", "name": "traces"}
2021-11-30T17:35:09.443-0800	info	zapgrpc/zapgrpc.go:174	[core] Subchannel picks a new address "otelcol:4317" to connect
2021-11-30T17:35:09.443-0800	info	builder/pipelines_builder.go:65	Pipeline is started.	{"name": "pipeline", "name": "traces"}
2021-11-30T17:35:09.443-0800	info	zapgrpc/zapgrpc.go:174	[core] pickfirstBalancer: UpdateSubConnState: 0xc0000f4440, {CONNECTING <nil>}
2021-11-30T17:35:09.443-0800	info	builder/pipelines_builder.go:54	Pipeline is starting...	{"name": "pipeline", "name": "metrics"}
2021-11-30T17:35:09.443-0800	info	zapgrpc/zapgrpc.go:174	[core] Channel Connectivity change to CONNECTING
2021-11-30T17:35:09.443-0800	info	builder/pipelines_builder.go:65	Pipeline is started.	{"name": "pipeline", "name": "metrics"}
2021-11-30T17:35:09.443-0800	info	service/service.go:101	Starting receivers...
2021-11-30T17:35:09.444-0800	info	builder/receivers_builder.go:68	Receiver is starting...	{"kind": "receiver", "name": "otlp"}
2021-11-30T17:35:09.444-0800	info	otlpreceiver/otlp.go:68	Starting GRPC server on endpoint 0.0.0.0:4317	{"kind": "receiver", "name": "otlp"}
2021-11-30T17:35:09.445-0800	info	builder/receivers_builder.go:73	Receiver started.	{"kind": "receiver", "name": "otlp"}
2021-11-30T17:35:09.445-0800	info	service/telemetry.go:92	Setting up own telemetry...
2021-11-30T17:35:09.449-0800	warn	zapgrpc/zapgrpc.go:191	[core] grpc: addrConn.createTransport failed to connect to {otelcol:4317 otelcol:4317 <nil> <nil> 0 <nil>}. Err: connection error: desc = "transport: Error while dialing dial tcp: lookup otelcol: no such host"
2021-11-30T17:35:09.449-0800	info	zapgrpc/zapgrpc.go:174	[core] Subchannel Connectivity change to TRANSIENT_FAILURE
2021-11-30T17:35:09.449-0800	info	zapgrpc/zapgrpc.go:174	[core] pickfirstBalancer: UpdateSubConnState: 0xc0000f4440, {TRANSIENT_FAILURE connection error: desc = "transport: Error while dialing dial tcp: lookup otelcol: no such host"}
2021-11-30T17:35:09.449-0800	info	zapgrpc/zapgrpc.go:174	[core] Channel Connectivity change to TRANSIENT_FAILURE
2021-11-30T17:35:09.455-0800	info	service/telemetry.go:116	Serving Prometheus metrics	{"address": ":8888", "level": "basic", "service.instance.id": "a579c69f-edb3-4fca-8911-e6fd9a5f5344", "service.version": "latest"}
2021-11-30T17:35:09.456-0800	info	service/collector.go:243	Starting cmd-otelcol...	{"Version": "0.40.0-dev", "NumCPU": 12}
2021-11-30T17:35:09.456-0800	info	service/collector.go:136	Everything is ready. Begin running and processing data.
2021-11-30T17:35:10.449-0800	info	zapgrpc/zapgrpc.go:174	[core] Subchannel Connectivity change to IDLE
2021-11-30T17:35:10.449-0800	info	zapgrpc/zapgrpc.go:174	[core] pickfirstBalancer: UpdateSubConnState: 0xc0000f4440, {IDLE connection error: desc = "transport: Error while dialing dial tcp: lookup otelcol: no such host"}
2021-11-30T17:35:10.449-0800	info	zapgrpc/zapgrpc.go:174	[core] Channel Connectivity change to IDLE
2021-11-30T17:35:12.072-0800	info	zapgrpc/zapgrpc.go:174	[transport] transport: loopyWriter.run returning. connection error: desc = "transport is closing"
2021-11-30T17:35:12.255-0800	INFO	loggingexporter/logging_exporter.go:40	TracesExporter	{"#spans": 2}
2021-11-30T17:35:28.595-0800	info	zapgrpc/zapgrpc.go:174	[core] Subchannel Connectivity change to TRANSIENT_FAILURE
2021-11-30T17:35:28.595-0800	info	zapgrpc/zapgrpc.go:174	[core] pickfirstBalancer: UpdateSubConnState: 0xc0000f4440, {TRANSIENT_FAILURE connection error: desc = "transport: Error while dialing dial tcp: lookup otelcol: no such host"}
2021-11-30T17:35:28.595-0800	info	zapgrpc/zapgrpc.go:174	[core] Channel Connectivity change to TRANSIENT_FAILURE
2021-11-30T17:35:28.595-0800	info	exporterhelper/queued_retry.go:215	Exporting failed. Will retry the request after interval.	{"kind": "exporter", "name": "otlp", "error": "rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp: lookup otelcol: no such host\"", "interval": "13.101300599s"}
2021-11-30T17:35:32.049-0800	info	zapgrpc/zapgrpc.go:174	[core] Subchannel Connectivity change to IDLE
2021-11-30T17:35:32.049-0800	info	zapgrpc/zapgrpc.go:174	[core] pickfirstBalancer: UpdateSubConnState: 0xc0000f4440, {IDLE connection error: desc = "transport: Error while dialing dial tcp: lookup otelcol: no such host"}
2021-11-30T17:35:32.049-0800	info	zapgrpc/zapgrpc.go:174	[core] Channel Connectivity change to IDLE

DEBUG in development mode

service:
  telemetry:
    logs:
      level: "DEBUG"
      development: true
      encoding: "console"
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging, otlp]

produces

2021-11-30T17:30:55.091-0800	info	builder/pipelines_builder.go:65	Pipeline is started.	{"name": "pipeline", "name": "metrics"}
2021-11-30T17:30:55.091-0800	info	builder/pipelines_builder.go:54	Pipeline is starting...	{"name": "pipeline", "name": "traces"}
2021-11-30T17:30:55.091-0800	info	zapgrpc/zapgrpc.go:174	[core] Subchannel picks a new address "otelcol:4317" to connect
2021-11-30T17:30:55.091-0800	info	builder/pipelines_builder.go:65	Pipeline is started.	{"name": "pipeline", "name": "traces"}
2021-11-30T17:30:55.091-0800	info	zapgrpc/zapgrpc.go:174	[core] pickfirstBalancer: UpdateSubConnState: 0xc00011c8f0, {CONNECTING <nil>}
2021-11-30T17:30:55.091-0800	info	service/service.go:101	Starting receivers...
2021-11-30T17:30:55.091-0800	info	zapgrpc/zapgrpc.go:174	[core] Channel Connectivity change to CONNECTING
2021-11-30T17:30:55.091-0800	info	builder/receivers_builder.go:68	Receiver is starting...	{"kind": "receiver", "name": "otlp"}
2021-11-30T17:30:55.091-0800	info	otlpreceiver/otlp.go:68	Starting GRPC server on endpoint 0.0.0.0:4317	{"kind": "receiver", "name": "otlp"}
2021-11-30T17:30:55.091-0800	info	builder/receivers_builder.go:73	Receiver started.	{"kind": "receiver", "name": "otlp"}
2021-11-30T17:30:55.091-0800	info	service/telemetry.go:92	Setting up own telemetry...
2021-11-30T17:30:55.101-0800	info	service/telemetry.go:116	Serving Prometheus metrics	{"address": ":8888", "level": "basic", "service.instance.id": "38ece474-2cdc-42a3-b442-46477936c2b7", "service.version": "latest"}
2021-11-30T17:30:55.101-0800	info	service/collector.go:243	Starting cmd-otelcol...	{"Version": "0.40.0-dev", "NumCPU": 12}
2021-11-30T17:30:55.101-0800	info	service/collector.go:136	Everything is ready. Begin running and processing data.
2021-11-30T17:30:55.236-0800	warn	zapgrpc/zapgrpc.go:191	[core] grpc: addrConn.createTransport failed to connect to {otelcol:4317 otelcol:4317 <nil> <nil> 0 <nil>}. Err: connection error: desc = "transport: Error while dialing dial tcp: lookup otelcol: no such host"
go.uber.org/zap/zapgrpc.(*Logger).Warningln
	go.uber.org/zap@v1.19.1/zapgrpc/zapgrpc.go:191
google.golang.org/grpc/internal/grpclog.WarningDepth
	google.golang.org/grpc@v1.42.0/internal/grpclog/grpclog.go:46
google.golang.org/grpc/grpclog.(*componentData).WarningDepth
	google.golang.org/grpc@v1.42.0/grpclog/component.go:41
google.golang.org/grpc/internal/channelz.Warningf
	google.golang.org/grpc@v1.42.0/internal/channelz/logging.go:75
google.golang.org/grpc.(*addrConn).createTransport
	google.golang.org/grpc@v1.42.0/clientconn.go:1315
google.golang.org/grpc.(*addrConn).tryAllAddrs
	google.golang.org/grpc@v1.42.0/clientconn.go:1249
google.golang.org/grpc.(*addrConn).resetTransport
	google.golang.org/grpc@v1.42.0/clientconn.go:1184
google.golang.org/grpc.(*addrConn).connect
	google.golang.org/grpc@v1.42.0/clientconn.go:845
2021-11-30T17:30:55.237-0800	info	zapgrpc/zapgrpc.go:174	[core] Subchannel Connectivity change to TRANSIENT_FAILURE
2021-11-30T17:30:55.237-0800	info	zapgrpc/zapgrpc.go:174	[core] pickfirstBalancer: UpdateSubConnState: 0xc00011c8f0, {TRANSIENT_FAILURE connection error: desc = "transport: Error while dialing dial tcp: lookup otelcol: no such host"}
2021-11-30T17:30:55.237-0800	info	zapgrpc/zapgrpc.go:174	[core] Channel Connectivity change to TRANSIENT_FAILURE
2021-11-30T17:30:56.238-0800	info	zapgrpc/zapgrpc.go:174	[core] Subchannel Connectivity change to IDLE
2021-11-30T17:30:56.238-0800	info	zapgrpc/zapgrpc.go:174	[core] pickfirstBalancer: UpdateSubConnState: 0xc00011c8f0, {IDLE connection error: desc = "transport: Error while dialing dial tcp: lookup otelcol: no such host"}
2021-11-30T17:30:56.238-0800	info	zapgrpc/zapgrpc.go:174	[core] Channel Connectivity change to IDLE
^C2021-11-30T17:30:58.448-0800	info	service/collector.go:167	Received signal from OS	{"signal": "interrupt"}
2021-11-30T17:30:58.448-0800	info	service/collector.go:259	Starting shutdown...
2021-11-30

Link to tracking Issue: #2237

Testing: Added unit tests and manual testing

- used grpc_settable instead of grpc_zap ReplaceV2 directly as it supports multiple sets in thread safe fashoin
  as opposed to grpc_zap.ReplaceV2
@pavankrish123 pavankrish123 requested review from a team and bogdandrutu December 1, 2021 21:43
@codecov
Copy link

codecov bot commented Dec 1, 2021

Codecov Report

Merging #4501 (c2f49c0) into main (9309a1f) will increase coverage by 0.02%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #4501      +/-   ##
==========================================
+ Coverage   90.67%   90.70%   +0.02%     
==========================================
  Files         179      179              
  Lines       10414    10445      +31     
==========================================
+ Hits         9443     9474      +31     
  Misses        755      755              
  Partials      216      216              
Impacted Files Coverage Δ
service/collector.go 74.21% <100.00%> (+0.32%) ⬆️
service/internal/telemetrylogs/logger.go 92.85% <100.00%> (+5.35%) ⬆️
exporter/otlphttpexporter/otlp.go 82.92% <0.00%> (+0.87%) ⬆️
exporter/otlpexporter/otlp.go 74.41% <0.00%> (+1.57%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9309a1f...c2f49c0. Read the comment docs.

go.sum Outdated Show resolved Hide resolved
@jpkrohling jpkrohling self-requested a review December 2, 2021 08:50
Copy link
Member

@jpkrohling jpkrohling left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like the right direction to me, we just need to settle on the opentracing dependency that is brought by go-grpc-middleware.

service/internal/telemetrylogs/logger.go Outdated Show resolved Hide resolved
service/internal/telemetrylogs/logger.go Outdated Show resolved Hide resolved
service/collector.go Outdated Show resolved Hide resolved
service/collector.go Outdated Show resolved Hide resolved
@pavankrish123
Copy link
Contributor Author

pavankrish123 commented Dec 10, 2021

@bogdandrutu @tigrannajaryan can we please get either of your blessing on this PR and let me know if anything else is needed. Please and Thank you.

@jpkrohling Also I do see release-after-ga label on the tracking feature request #2237. Let me know if that label is still valid and we want to merge according to that label.

I can update our internal teams using collector with a workaround solution involving environment variables accordingly.

Thanks you

cc: @svrnm

@jpkrohling
Copy link
Member

I don't think that label has any meaning by now, don't worry about it.

@bogdandrutu, @tigrannajaryan, we need a maintainer to merge this, possibly doing a final review (despite having three approvals).

@tigrannajaryan
Copy link
Member

tigrannajaryan commented Dec 13, 2021

  • In the case of collector default configuration, gRPC logs minimum level is set to WARN (to reduce the log traffic) but shared rest of collector's logger configuration.

Before merging this I would like to understand how chatty is gRPC in WARN level. For example if there is no connection to the destination does it periodically print warnings? Can this flood the logs?

@pavankrish123
Copy link
Contributor Author

pavankrish123 commented Dec 13, 2021

Thank you @tigrannajaryan - In case of default collector setting - which sets gRPC logging to WARN - We will have 1 message per connection failure per retry for exporter.

2021-11-30T16:47:38.384-0800    warn    zapgrpc/zapgrpc.go:191  [core] grpc: addrConn.createTransport failed to connect to {otelcol:4317 otelcol:4317 <nil> <nil> 0 <nil>}. Err: connection error: desc = "transport: Error while dialing dial tcp: lookup otelcol: no such host"

I have not tested for all other failures - but quickly glancing the gRPC code. The error specific verbose logs only happen to appear in INFO level.

On the other hand - once we set debug logging on collector - we will have more chatty messages about gRPC transport
these are 1 time start up messages per grpc exporter

2021-11-30T17:35:09.442-0800    info    zapgrpc/zapgrpc.go:174  [core] original dial target is: "otelcol:4317"
2021-11-30T17:35:09.443-0800    info    zapgrpc/zapgrpc.go:174  [core] parsed dial target is: {Scheme:otelcol Authority: Endpoint:4317 URL:{Scheme:otelcol Opaque:4317 User: Host: Path: RawPath: ForceQuery:false RawQuery: Fragment: RawFragment:}}
2021-11-30T17:35:09.443-0800    info    zapgrpc/zapgrpc.go:174  [core] fallback to scheme "passthrough"
2021-11-30T17:35:09.443-0800    info    zapgrpc/zapgrpc.go:174  [core] parsed dial target is: {Scheme:passthrough Authority: Endpoint:otelcol:4317 URL:{Scheme:passthrough Opaque: User: Host: Path:/otelcol:4317 RawPath: ForceQuery:false RawQuery: Fragment: RawFragment:}}
2021-11-30T17:35:09.443-0800    info    zapgrpc/zapgrpc.go:174  [core] Channel authority set to "otelcol:4317"
2021-11-30T17:35:09.443-0800    info    zapgrpc/zapgrpc.go:174  [core] ccResolverWrapper: sending update to cc: {[{otelcol:4317  <nil> <nil> 0 <nil>}] <nil> <nil>}
2021-11-30T17:35:09.443-0800    info    zapgrpc/zapgrpc.go:174  [core] ClientConn switching balancer to "pick_first"
2021-11-30T17:35:09.443-0800    info    zapgrpc/zapgrpc.go:174  [core] Channel switches to new LB policy "pick_first"

plus in case of failure (for no host error) we will have these set of error specific messages per retry cycle of the exporter.

2021-11-30T17:35:09.445-0800    info    service/telemetry.go:92 Setting up own telemetry...
2021-11-30T17:35:09.449-0800    warn    zapgrpc/zapgrpc.go:191  [core] grpc: addrConn.createTransport failed to connect to {otelcol:4317 otelcol:4317 <nil> <nil> 0 <nil>}. Err: connection error: desc = "transport: Error while dialing dial tcp: lookup otelcol: no such host"
2021-11-30T17:35:09.449-0800    info    zapgrpc/zapgrpc.go:174  [core] Subchannel Connectivity change to TRANSIENT_FAILURE
2021-11-30T17:35:09.449-0800    info    zapgrpc/zapgrpc.go:174  [core] pickfirstBalancer: UpdateSubConnState: 0xc0000f4440, {TRANSIENT_FAILURE connection error: desc = "transport: Error while dialing dial tcp: lookup otelcol: no such host"}
2021-11-30T17:35:09.449-0800    info    zapgrpc/zapgrpc.go:174  [core] Channel Connectivity change to TRANSIENT_FAILURE
2021-11-30T17:35:10.449-0800    info    zapgrpc/zapgrpc.go:174  [core] Subchannel Connectivity change to IDLE
2021-11-30T17:35:10.449-0800    info    zapgrpc/zapgrpc.go:174  [core] pickfirstBalancer: UpdateSubConnState: 0xc0000f4440, {IDLE connection error: desc = "transport: Error while dialing dial tcp: lookup otelcol: no such host"}
2021-11-30T17:35:10.449-0800    info    zapgrpc/zapgrpc.go:174  [core] Channel Connectivity change to IDLE
2021-11-30T17:35:12.072-0800    info    zapgrpc/zapgrpc.go:174  [transport] transport: loopyWriter.run returning. connection error: desc = "transport is closing"
2021-11-30T17:35:12.255-0800    INFO    loggingexporter/logging_exporter.go:40  TracesExporter  {"#spans": 2}
2021-11-30T17:35:28.595-0800    info    zapgrpc/zapgrpc.go:174  [core] Subchannel Connectivity change to TRANSIENT_FAILURE
2021-11-30T17:35:28.595-0800    info    zapgrpc/zapgrpc.go:174  [core] pickfirstBalancer: UpdateSubConnState: 0xc0000f4440, {TRANSIENT_FAILURE connection error: desc = "transport: Error while dialing dial tcp: lookup otelcol: no such host"}
2021-11-30T17:35:28.595-0800    info    zapgrpc/zapgrpc.go:174  [core] Channel Connectivity change to TRANSIENT_FAILURE

The default behavior of gRPC framework itself is to set logging level to ERR and write logs to `ioutil.DIscard.

@tigrannajaryan
Copy link
Member

This sounds ok to me. Let's merge this but be prepared to adjust the default behavior if it turns out to be more noisy than we would want.

@pavankrish123
Copy link
Contributor Author

Thanks @tigrannajaryan . Please let me know if you need anything else from my end.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants