: and / not allowed in pipeline configuration through CONFIG_STRING environment variable in 8.15.1

[tomudding]

Logstash information:

We are using the Docker image of Logstash on version 8.15.1.

JVM (e.g. java -version):

The one bundled in the Docker image.

OS version (uname -a if on a Unix-like system):

We have tested this on Docker standalone 26.1.4, standalone 27.1.2, and swarm 26.1.4. The actual machines that run the different Docker instances:

Linux 5.15.0-118-generic #128-Ubuntu SMP Fri Jul 5 09:28:59 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Linux 5.4.0-193-generic #213-Ubuntu SMP Fri Aug 2 19:14:16 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Description of the problem including expected versus actual behavior:

We use the Logstash container to ingest through Beats and output to ElasticSearch. However, when upgrading from version 8.15.0 to 8.15.1 our pipeline configuration became invalid when supplying it through the CONFIG_STRING environment variable in our docker-compose.yml.

Our configuration (works on 8.15.0):

input {
  stdin { }
  beats { port => 5040 }
}

output {
  elasticsearch {
    hosts => ["https://es:9200"]
    ssl_certificate_authorities => ["/usr/share/logstash/config/certs/ca.crt"]
    data_stream => "true"
    data_stream_type => "logs"
    user => "removed"
    password => "removed"
  }
  stdout { }
}

But on 8.15.1 we get the following error:

Failed to execute action {
  :action=>LogStash::PipelineAction::Create/pipeline_id:main,
  :exception=>"LogStash::ConfigurationError",
  :message=>"Expected one of [A-Za-z0-9_-], [ \\t\\r\\n], \"#\", \"{\", [A-Za-z0-9_], \",\", \"]\" at line 7, column 20 (byte 94) after output {\n  elasticsearch {\n    hosts => [http",
  :backtrace=>[
    "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'",
    "org/logstash/execution/AbstractPipelineExt.java:239:in `initialize'",
    "org/logstash/execution/AbstractPipelineExt.java:173:in `initialize'",
    "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:48:in `initialize'",
    "org/jruby/RubyClass.java:949:in `new'",
    "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:49:in `execute'",
    "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:386:in `block in converge_state'"
  ]
}

The character at column 20 is the first : in our hosts => ["https://es:9200"]. Eliminating the https:// does not work, as it then fails on the : specifying the port. Completely removing the port breaks on the first / in the ssl_certificate_authorities.

To ensure that this is not an issue with logstash-output-elasticsearch we also tested with the standard HTTP output plugin using the following config:

input {
  stdin { }
  beats { port => 5040 }
}

output {
  http {
    url => "https://otherhost:9200"
    http_method => "post"
  }
  stdout { }
}

We get a similar error for the : in the url:

Failed to execute action {
  :action=>LogStash::PipelineAction::Create/pipeline_id:main,
  :exception=>"LogStash::ConfigurationError",
  :message=>"Expected one of [A-Za-z0-9_-], [ \\t\\r\\n], \"#\", \"{\", [A-Za-z0-9_], \"}\" at line 7, column 17 (byte 82) after output {\n  http {\n    url => https",
  :backtrace=>[
    "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'",
    "org/logstash/execution/AbstractPipelineExt.java:239:in `initialize'",
    "org/logstash/execution/AbstractPipelineExt.java:173:in `initialize'",
    "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:48:in `initialize'",
    "org/jruby/RubyClass.java:949:in `new'",
    "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:49:in `execute'",
    "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:386:in `block in converge_state'"
  ]
}

---

Providing either pipeline config through logstash.conf (or another file in /usr/share/logstash/pipeline) does work as expected. As such, we believe there is a change in the parsing of the environment variable between 8.15.0 and 8.15.1.

[mashhurs]

[Editing after figuring out.]

I have reproduced the case by creating artifacts locally and ensured that #16365 introduced the regression

# example command
docker run --rm --name=local-ls-docker -p 9600:9600 -e CONFIG_STRING="input {
  stdin { }
  beats { port => 5040 }
}
output {
  elasticsearch {
    hosts => ["https://es:9200"]
    data_stream => "true"
    data_stream_type => "logs"
    user => "removed"
    password => "removed"
  }
  stdout { }
}" -it docker.elastic.co/logstash/logstash:8.15

# ERROR
[2024-09-12T04:49:38,228][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [A-Za-z0-9_-], [ \\t\\r\\n], \"#\", \"{\", [A-Za-z0-9_], \",\", \"]\" at line 7, column 20 (byte 94) after output {\n  elasticsearch {\n    hosts => [https", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:239:in `initialize'", "org/logstash/execution/AbstractPipelineExt.java:173:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:48:in `initialize'", "org/jruby/RubyClass.java:931:in `new'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:49:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:386:in `block in converge_state'"]}

I have captured the debug logs:

2024-09-11 22:04:30 Replacement: "input {   stdin { }   beats { port => 5040 } } output {   elasticsearch {     hosts => [\"https://es:9200\"]     data_stream => \"true\"     data_stream_type => \"logs\"     user => \"removed\"     password => \"removed\"   }   stdout { } }"
2024-09-11 22:04:30 Refined value: "input {   stdin { }   beats { port => 5040 } } output {   elasticsearch {     hosts => [https://es:9200]     data_stream => true     data_stream_type => logs     user => removed     password => removed   }   stdout { } }"

and confirming that refined value produced by refined_value = placeholder_value.gsub(/[\\"\\']/, '') causes the issue.