Using bundled JDK: /usr/share/logstash/jdk
Warning: no jvm.options file found.
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO ] 2022-11-30 09:51:01.228 [main] runner - Starting Logstash {"logstash.version"=>"8.4.0", "jruby.version"=>"jruby 9.3.6.0 (2.6.8) 2022-06-27 7a2cbcd376 OpenJDK 64-Bit Server VM 17.0.4+8 on 17.0.4+8 +jit [x86_64-linux]"}
[INFO ] 2022-11-30 09:51:01.235 [main] runner - JVM bootstrap flags: [-Dls.cgroup.cpuacct.path.override=/, -Dls.cgroup.cpu.path.override=/, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]
[INFO ] 2022-11-30 09:51:01.242 [main] settings - Creating directory {:setting=>"path.queue", :path=>"/usr/share/logstash/data/queue"}
[INFO ] 2022-11-30 09:51:01.243 [main] settings - Creating directory {:setting=>"path.dead_letter_queue", :path=>"/usr/share/logstash/data/dead_letter_queue"}
[WARN ] 2022-11-30 09:51:01.430 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2022-11-30 09:51:01.441 [LogStash::Runner] agent - No persistent UUID file found. Generating new UUID {:uuid=>"ff9f96b2-ab2a-490f-8e94-4fb1b7f4915d", :path=>"/usr/share/logstash/data/uuid"}
[INFO ] 2022-11-30 09:51:01.745 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[INFO ] 2022-11-30 09:51:01.975 [Converge PipelineAction::Create<main>] Reflections - Reflections took 123 ms to scan 1 urls, producing 125 keys and 434 values
[INFO ] 2022-11-30 09:51:02.131 [Converge PipelineAction::Create<main>] json - ECS compatibility is enabled but `target` option was not specified. This may cause fields to be set at the top-level of the event where they are likely to clash with the Elastic Common Schema. It is recommended to set the `target` option to avoid potential schema conflicts (if your data is ECS compliant or non-conflicting, feel free to ignore this message)
[INFO ] 2022-11-30 09:51:03.119 [Converge PipelineAction::Create<main>] javapipeline - Pipeline `main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
[INFO ] 2022-11-30 09:51:03.124 [[main]-pipeline-manager] opensearch - New OpenSearch output {:class=>"LogStash::Outputs::OpenSearch", :hosts=>["//10.98.55.121:9200"]}
[INFO ] 2022-11-30 09:51:03.243 [[main]-pipeline-manager] opensearch - OpenSearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://admin:xxxxxx@10.98.55.121:9200/]}}
[WARN ] 2022-11-30 09:51:03.303 [[main]-pipeline-manager] opensearch - Attempted to resurrect connection to dead OpenSearch instance, but got an error {:url=>"http://admin:xxxxxx@10.98.55.121:9200/", :exception=>LogStash::Outputs::OpenSearch::HttpClient::Pool::HostUnreachableError, :message=>"OpenSearch Unreachable: [http://admin:xxxxxx@10.98.55.121:9200/][Manticore::ClientProtocolException] 10.98.55.121:9200 failed to respond"}
[INFO ] 2022-11-30 09:51:03.318 [[main]-pipeline-manager] javapipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>3, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>375, "pipeline.sources"=>["/usr/share/logstash/pipeline/logstash.conf"], :thread=>"#<Thread:0x5331f9bc@/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:130 run>"}
[INFO ] 2022-11-30 09:51:03.947 [[main]-pipeline-manager] javapipeline - Pipeline Java execution initialization time {"seconds"=>0.63}
[INFO ] 2022-11-30 09:51:03.950 [[main]-pipeline-manager] javapipeline - Pipeline started {"pipeline.id"=>"main"}
[INFO ] 2022-11-30 09:51:03.962 [Agent thread] agent - Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[WARN ] 2022-11-30 09:51:04.015 [[main]<kafka] CommonClientConfigs - Configuration 'client.dns.lookup' with value 'default' is deprecated and will be removed in future version. Please use 'use_all_dns_ips' or another non-deprecated value.
[INFO ] 2022-11-30 09:51:04.016 [[main]<kafka] ConsumerConfig - ConsumerConfig values:
allow.auto.create.topics = true
auto.commit.interval.ms = 5000
auto.offset.reset = latest
bootstrap.servers = [10.111.178.24:9092]
check.crcs = true
client.dns.lookup = default
client.id = logstash-0
client.rack =
connections.max.idle.ms = 540000
default.api.timeout.ms = 60000
enable.auto.commit = true
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id = logstash
group.instance.id = null
heartbeat.interval.ms = 3000
interceptor.classes = []
internal.leave.group.on.close = true
internal.throw.on.fetch.stable.offset.unsupported = false
isolation.level = read_uncommitted
key.deserializer = class org.apache.kafka.common.serialization.StringDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 300000
max.poll.records = 500
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
receive.buffer.bytes = 32768
reconnect.backoff.max.ms = 50
reconnect.backoff.ms = 50
request.timeout.ms = 40000
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
security.providers = null
send.buffer.bytes = 131072
session.timeout.ms = 10000
socket.connection.setup.timeout.max.ms = 30000
socket.connection.setup.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.3]
ssl.endpoint.identification.algorithm = https
ssl.engine.factory.class = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.certificate.chain = null
ssl.keystore.key = null
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLSv1.3
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.certificates = null
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer
[INFO ] 2022-11-30 09:51:04.104 [[main]<kafka] AppInfoParser - Kafka version: 2.8.1
[INFO ] 2022-11-30 09:51:04.104 [[main]<kafka] AppInfoParser - Kafka commitId: 839b886f9b732b15
[INFO ] 2022-11-30 09:51:04.105 [[main]<kafka] AppInfoParser - Kafka startTimeMs: 1669801864102
[INFO ] 2022-11-30 09:51:04.108 [[main]<kafka] KafkaConsumer - [Consumer clientId=logstash-0, groupId=logstash] Subscribed to topic(s): t_events
[INFO ] 2022-11-30 09:51:04.126 [kafka-input-worker-logstash-0] json - ECS compatibility is enabled but `target` option was not specified. This may cause fields to be set at the top-level of the event where they are likely to clash with the Elastic Common Schema. It is recommended to set the `target` option to avoid potential schema conflicts (if your data is ECS compliant or non-conflicting, feel free to ignore this message)
[INFO ] 2022-11-30 09:51:04.484 [kafka-input-worker-logstash-0] Metadata - [Consumer clientId=logstash-0, groupId=logstash] Cluster ID: -So-qzioTg6u0XDu6rFIYg
[INFO ] 2022-11-30 09:51:04.485 [kafka-input-worker-logstash-0] AbstractCoordinator - [Consumer clientId=logstash-0, groupId=logstash] Discovered group coordinator my-cluster-kafka-1.my-cluster-kafka-brokers.dv-poc.svc:9092 (id: 2147483646 rack: null)
[INFO ] 2022-11-30 09:51:04.492 [kafka-input-worker-logstash-0] AbstractCoordinator - [Consumer clientId=logstash-0, groupId=logstash] (Re-)joining group
[INFO ] 2022-11-30 09:51:04.526 [kafka-input-worker-logstash-0] AbstractCoordinator - [Consumer clientId=logstash-0, groupId=logstash] (Re-)joining group
[INFO ] 2022-11-30 09:51:07.554 [kafka-input-worker-logstash-0] AbstractCoordinator - [Consumer clientId=logstash-0, groupId=logstash] Successfully joined group with generation Generation{generationId=1, memberId='logstash-0-a7df73a4-2046-4ca4-8176-36c2de4b4e7b', protocol='range'}
[INFO ] 2022-11-30 09:51:07.557 [kafka-input-worker-logstash-0] ConsumerCoordinator - [Consumer clientId=logstash-0, groupId=logstash] Finished assignment for group at generation 1: {logstash-0-a7df73a4-2046-4ca4-8176-36c2de4b4e7b=Assignment(partitions=[t_events-0])}
[INFO ] 2022-11-30 09:51:07.651 [kafka-input-worker-logstash-0] AbstractCoordinator - [Consumer clientId=logstash-0, groupId=logstash] Successfully synced group in generation Generation{generationId=1, memberId='logstash-0-a7df73a4-2046-4ca4-8176-36c2de4b4e7b', protocol='range'}
[INFO ] 2022-11-30 09:51:07.651 [kafka-input-worker-logstash-0] ConsumerCoordinator - [Consumer clientId=logstash-0, groupId=logstash] Notifying assignor about the new Assignment(partitions=[t_events-0])
[INFO ] 2022-11-30 09:51:07.654 [kafka-input-worker-logstash-0] ConsumerCoordinator - [Consumer clientId=logstash-0, groupId=logstash] Adding newly assigned partitions: t_events-0
[INFO ] 2022-11-30 09:51:07.673 [kafka-input-worker-logstash-0] ConsumerCoordinator - [Consumer clientId=logstash-0, groupId=logstash] Found no committed offset for partition t_events-0
[INFO ] 2022-11-30 09:51:07.693 [kafka-input-worker-logstash-0] SubscriptionState - [Consumer clientId=logstash-0, groupId=logstash] Resetting offset for partition t_events-0 to position FetchPosition{offset=1480, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[my-cluster-kafka-0.my-cluster-kafka-brokers.dv-poc.svc:9092 (id: 0 rack: null)], epoch=0}}.
[WARN ] 2022-11-30 09:51:08.320 [Ruby-0-Thread-9: /usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-output-opensearch-2.0.0-java/lib/logstash/outputs/opensearch/http_client/pool.rb:217] opensearch - Attempted to resurrect connection to dead OpenSearch instance, but got an error {:url=>"http://admin:xxxxxx@10.98.55.121:9200/", :exception=>LogStash::Outputs::OpenSearch::HttpClient::Pool::HostUnreachableError, :message=>"OpenSearch Unreachable: [http://admin:xxxxxx@10.98.55.121:2300/][Manticore::ClientProtocolException] 10.98.55.121:9200 failed to respond"}
[WARN ] 2022-11-30 09:51:13.332 [Ruby-0-Thread-9: /usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-output-opensearch-2.0.0-java/lib/logstash/outputs/opensearch/http_client/pool.rb:217] opensearch - Attempted to resurrect connection to dead OpenSearch instance, but got an error {:url=>"http://admin:xxxxxx@10.98.55.121:9200/", :exception=>LogStash::Outputs::OpenSearch::HttpClient::Pool::HostUnreachableError, :message=>"OpenSearch Unreachable: [http://admin:xxxxxx@10.98.55.121:9200/][Manticore::ClientProtocolException] 10.98.55.121:9200 failed to respond"}
[WARN ] 2022-11-30 09:51:18.344 [Ruby-0-Thread-9: /usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-output-opensearch-2.0.0-java/lib/logstash/outputs/opensearch/http_client/pool.rb:217] opensearch - Attempted to resurrect connection to dead OpenSearch instance, but got an error {:url=>"http://admin:xxxxxx@10.98.55.121:9200/", :exception=>LogStash::Outputs::OpenSearch::HttpClient::Pool::HostUnreachableError, :message=>"OpenSearch Unreachable: [http://admin:xxxxxx@10.98.55.121:9200/][Manticore::ClientProtocolException] 10.98.55.121:9200 failed to respond"}