geisbruch/jaai-api

Use chat gpt with your own data

jaai-api's Issues

docker compose fails

Running docker-compose up I got this:

✔ Container jaai-api_postgres_1 Recreated 0.1s ✔ Container jaai-api_opensearch_1 Recreated 0.1s ✔ Container jaai-api_api_1 Recreated 0.1s Attaching to jaai-api-api-1, jaai-api-opensearch-1, jaai-api-postgres-1 jaai-api-opensearch-1 | Enabling execution of install_demo_configuration.sh for OpenSearch Security Plugin jaai-api-opensearch-1 | ************************************************************************** jaai-api-opensearch-1 | ** This tool will be deprecated in the next major release of OpenSearch ** jaai-api-opensearch-1 | ** https://github.com/opensearch-project/security/issues/1755 ** jaai-api-opensearch-1 | ************************************************************************** jaai-api-opensearch-1 | OpenSearch Security Demo Installer jaai-api-opensearch-1 | ** Warning: Do not use on production or public reachable systems ** jaai-api-opensearch-1 | Basedir: /usr/share/opensearch jaai-api-opensearch-1 | OpenSearch install type: rpm/deb on NAME="Amazon Linux" jaai-api-opensearch-1 | OpenSearch config dir: /usr/share/opensearch/config jaai-api-opensearch-1 | OpenSearch config file: /usr/share/opensearch/config/opensearch.yml jaai-api-opensearch-1 | OpenSearch bin dir: /usr/share/opensearch/bin jaai-api-opensearch-1 | OpenSearch plugins dir: /usr/share/opensearch/plugins jaai-api-opensearch-1 | OpenSearch lib dir: /usr/share/opensearch/lib jaai-api-opensearch-1 | Detected OpenSearch Version: x-content-2.6.0 jaai-api-opensearch-1 | Detected OpenSearch Security Version: 2.6.0.0 jaai-api-opensearch-1 | jaai-api-opensearch-1 | ### Success jaai-api-opensearch-1 | ### Execute this script now on all your nodes and then start all nodes jaai-api-opensearch-1 | ### OpenSearch Security will be automatically initialized. jaai-api-opensearch-1 | ### If you like to change the runtime configuration jaai-api-opensearch-1 | ### change the files in ../../../config/opensearch-security and execute: jaai-api-opensearch-1 | "/usr/share/opensearch/plugins/opensearch-security/tools/securityadmin.sh" -cd "/usr/share/opensearch/config/opensearch-security" -icl -key "/usr/share/opensearch/config/kirk-key.pem" -cert "/usr/share/opensearch/config/kirk.pem" -cacert "/usr/share/opensearch/config/root-ca.pem" -nhnv jaai-api-opensearch-1 | ### or run ./securityadmin_demo.sh jaai-api-opensearch-1 | ### To use the Security Plugin ConfigurationGUI jaai-api-opensearch-1 | ### To access your secured cluster open https://<hostname>:<HTTP port> and log in with admin/admin. jaai-api-opensearch-1 | ### (Ignore the SSL certificate warning because we installed self-signed demo certificates) jaai-api-opensearch-1 | Enabling OpenSearch Security Plugin jaai-api-opensearch-1 | Enabling execution of OPENSEARCH_HOME/bin/opensearch-performance-analyzer/performance-analyzer-agent-cli for OpenSearch Performance Analyzer Plugin jaai-api-api-1 | psql: error: connection to server at "postgres" (172.21.0.2), port 5432 failed: Connection refused jaai-api-api-1 | Is the server running on that host and accepting TCP/IP connections? jaai-api-api-1 | Checking postgres jaai-api-api-1 | psql: error: connection to server at "postgres" (172.21.0.2), port 5432 failed: Connection refused jaai-api-api-1 | Is the server running on that host and accepting TCP/IP connections? jaai-api-api-1 | Checking postgres jaai-api-postgres-1 | jaai-api-postgres-1 | PostgreSQL Database directory appears to contain a database; Skipping initialization jaai-api-postgres-1 | jaai-api-postgres-1 | 2023-04-13 18:21:50.018 UTC [1] LOG: starting PostgreSQL 14.2 (Debian 14.2-1.pgdg110+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit jaai-api-postgres-1 | 2023-04-13 18:21:50.019 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 jaai-api-postgres-1 | 2023-04-13 18:21:50.019 UTC [1] LOG: listening on IPv6 address "::", port 5432 jaai-api-postgres-1 | 2023-04-13 18:21:50.022 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" jaai-api-postgres-1 | 2023-04-13 18:21:50.037 UTC [26] LOG: database system was interrupted; last known up at 2023-04-13 18:12:20 UTC jaai-api-postgres-1 | 2023-04-13 18:21:50.136 UTC [27] FATAL: the database system is starting up jaai-api-api-1 | psql: error: connection to server at "postgres" (172.21.0.2), port 5432 failed: FATAL: the database system is starting up jaai-api-api-1 | Checking postgres jaai-api-opensearch-1 | WARNING: A terminally deprecated method in java.lang.System has been called jaai-api-opensearch-1 | WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/opensearch/lib/opensearch-2.6.0.jar) jaai-api-opensearch-1 | WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch jaai-api-opensearch-1 | WARNING: System::setSecurityManager will be removed in a future release jaai-api-postgres-1 | 2023-04-13 18:21:51.146 UTC [28] FATAL: the database system is starting up jaai-api-api-1 | psql: error: connection to server at "postgres" (172.21.0.2), port 5432 failed: FATAL: the database system is starting up jaai-api-api-1 | Checking postgres jaai-api-postgres-1 | 2023-04-13 18:21:51.225 UTC [26] LOG: database system was not properly shut down; automatic recovery in progress jaai-api-postgres-1 | 2023-04-13 18:21:51.244 UTC [26] LOG: redo starts at 0/16FBC60 jaai-api-postgres-1 | 2023-04-13 18:21:51.244 UTC [26] LOG: invalid record length at 0/16FBD48: wanted 24, got 0 jaai-api-postgres-1 | 2023-04-13 18:21:51.244 UTC [26] LOG: redo done at 0/16FBD10 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s jaai-api-postgres-1 | 2023-04-13 18:21:51.260 UTC [1] LOG: database system is ready to accept connections jaai-api-opensearch-1 | WARNING: A terminally deprecated method in java.lang.System has been called jaai-api-opensearch-1 | WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/opensearch/lib/opensearch-2.6.0.jar) jaai-api-opensearch-1 | WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security jaai-api-opensearch-1 | WARNING: System::setSecurityManager will be removed in a future release jaai-api-opensearch-1 | [2023-04-13T18:21:51,427][INFO ][o.o.n.Node ] [972572e0721a] version[2.6.0], pid[102], build[tar/7203a5af21a8a009aece1474446b437a3c674db6/2023-02-24T18:58:37.352296474Z], OS[Linux/5.15.49-linuxkit/aarch64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/17.0.6/17.0.6+10] jaai-api-opensearch-1 | [2023-04-13T18:21:51,428][INFO ][o.o.n.Node ] [972572e0721a] JVM home [/usr/share/opensearch/jdk], using bundled JDK [true] jaai-api-opensearch-1 | [2023-04-13T18:21:51,428][INFO ][o.o.n.Node ] [972572e0721a] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/opensearch-5571368306793131649, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Dclk.tck=100, -Djdk.attach.allowAttachSelf=true, -Djava.security.policy=/usr/share/opensearch/config/opensearch-performance-analyzer/opensearch_security.policy, --add-opens=jdk.attach/sun.tools.attach=ALL-UNNAMED, -Dopensearch.cgroups.hierarchy.override=/, -XX:MaxDirectMemorySize=536870912, -Dopensearch.path.home=/usr/share/opensearch, -Dopensearch.path.conf=/usr/share/opensearch/config, -Dopensearch.distribution.type=tar, -Dopensearch.bundled_jdk=true] jaai-api-opensearch-1 | [2023-04-13T18:21:52,041][WARN ][stderr ] [972572e0721a] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". jaai-api-opensearch-1 | [2023-04-13T18:21:52,041][WARN ][stderr ] [972572e0721a] SLF4J: Defaulting to no-operation (NOP) logger implementation jaai-api-opensearch-1 | [2023-04-13T18:21:52,041][WARN ][stderr ] [972572e0721a] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. jaai-api-opensearch-1 | [2023-04-13T18:21:52,048][INFO ][o.o.s.s.t.SSLConfig ] [972572e0721a] SSL dual mode is disabled jaai-api-opensearch-1 | [2023-04-13T18:21:52,048][WARN ][o.o.s.OpenSearchSecurityPlugin] [972572e0721a] OpenSearch Security plugin installed but disabled. This can expose your configuration (including passwords) to the public. jaai-api-api-1 | ?column? jaai-api-api-1 | ---------- jaai-api-api-1 | 1 jaai-api-api-1 | (1 row) jaai-api-api-1 | jaai-api-postgres-1 | 2023-04-13 18:21:52.294 UTC [36] ERROR: database "api" already exists jaai-api-postgres-1 | 2023-04-13 18:21:52.294 UTC [36] STATEMENT: CREATE DATABASE api; jaai-api-api-1 | ERROR: database "api" already exists jaai-api-api-1 | NOTICE: schema "test" already exists, skipping jaai-api-api-1 | CREATE SCHEMA jaai-api-opensearch-1 | [2023-04-13T18:21:52,622][INFO ][o.o.p.c.PluginSettings ] [972572e0721a] Config: metricsLocation: /dev/shm/performanceanalyzer/, metricsDeletionInterval: 1, httpsEnabled: false, cleanup-metrics-db-files: true, batch-metrics-retention-period-minutes: 7, rpc-port: 9650, webservice-port 9600 jaai-api-opensearch-1 | [2023-04-13T18:21:53,041][INFO ][o.o.i.r.ReindexPlugin ] [972572e0721a] ReindexPlugin reloadSPI called jaai-api-opensearch-1 | [2023-04-13T18:21:53,042][INFO ][o.o.i.r.ReindexPlugin ] [972572e0721a] Unable to find any implementation for RemoteReindexExtension jaai-api-opensearch-1 | [2023-04-13T18:21:53,084][INFO ][o.o.j.JobSchedulerPlugin ] [972572e0721a] Loaded scheduler extension: reports-scheduler, index: .opendistro-reports-definitions jaai-api-opensearch-1 | [2023-04-13T18:21:53,086][INFO ][o.o.j.JobSchedulerPlugin ] [972572e0721a] Loaded scheduler extension: opendistro_anomaly_detector, index: .opendistro-anomaly-detector-jobs jaai-api-opensearch-1 | [2023-04-13T18:21:53,086][INFO ][o.o.j.JobSchedulerPlugin ] [972572e0721a] Loaded scheduler extension: opendistro-index-management, index: .opendistro-ism-config jaai-api-opensearch-1 | [2023-04-13T18:21:53,105][INFO ][o.o.j.JobSchedulerPlugin ] [972572e0721a] Loaded scheduler extension: observability, index: .opensearch-observability-job jaai-api-opensearch-1 | [2023-04-13T18:21:53,109][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded module [aggs-matrix-stats] jaai-api-opensearch-1 | [2023-04-13T18:21:53,109][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded module [analysis-common] jaai-api-opensearch-1 | [2023-04-13T18:21:53,109][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded module [geo] jaai-api-opensearch-1 | [2023-04-13T18:21:53,109][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded module [ingest-common] jaai-api-opensearch-1 | [2023-04-13T18:21:53,109][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded module [ingest-geoip] jaai-api-opensearch-1 | [2023-04-13T18:21:53,109][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded module [ingest-user-agent] jaai-api-opensearch-1 | [2023-04-13T18:21:53,109][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded module [lang-expression] jaai-api-opensearch-1 | [2023-04-13T18:21:53,109][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded module [lang-mustache] jaai-api-opensearch-1 | [2023-04-13T18:21:53,109][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded module [lang-painless] jaai-api-opensearch-1 | [2023-04-13T18:21:53,109][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded module [mapper-extras] jaai-api-opensearch-1 | [2023-04-13T18:21:53,109][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded module [opensearch-dashboards] jaai-api-opensearch-1 | [2023-04-13T18:21:53,109][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded module [parent-join] jaai-api-opensearch-1 | [2023-04-13T18:21:53,109][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded module [percolator] jaai-api-opensearch-1 | [2023-04-13T18:21:53,109][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded module [rank-eval] jaai-api-opensearch-1 | [2023-04-13T18:21:53,109][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded module [reindex] jaai-api-opensearch-1 | [2023-04-13T18:21:53,109][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded module [repository-url] jaai-api-opensearch-1 | [2023-04-13T18:21:53,109][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded module [systemd] jaai-api-opensearch-1 | [2023-04-13T18:21:53,109][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded module [transport-netty4] jaai-api-opensearch-1 | [2023-04-13T18:21:53,109][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded plugin [opensearch-alerting] jaai-api-opensearch-1 | [2023-04-13T18:21:53,110][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded plugin [opensearch-anomaly-detection] jaai-api-opensearch-1 | [2023-04-13T18:21:53,110][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded plugin [opensearch-asynchronous-search] jaai-api-opensearch-1 | [2023-04-13T18:21:53,110][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded plugin [opensearch-cross-cluster-replication] jaai-api-opensearch-1 | [2023-04-13T18:21:53,110][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded plugin [opensearch-geospatial] jaai-api-opensearch-1 | [2023-04-13T18:21:53,110][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded plugin [opensearch-index-management] jaai-api-opensearch-1 | [2023-04-13T18:21:53,110][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded plugin [opensearch-job-scheduler] jaai-api-opensearch-1 | [2023-04-13T18:21:53,110][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded plugin [opensearch-knn] jaai-api-opensearch-1 | [2023-04-13T18:21:53,110][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded plugin [opensearch-ml] jaai-api-opensearch-1 | [2023-04-13T18:21:53,110][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded plugin [opensearch-neural-search] jaai-api-opensearch-1 | [2023-04-13T18:21:53,110][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded plugin [opensearch-notifications] jaai-api-opensearch-1 | [2023-04-13T18:21:53,110][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded plugin [opensearch-notifications-core] jaai-api-opensearch-1 | [2023-04-13T18:21:53,110][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded plugin [opensearch-observability] jaai-api-opensearch-1 | [2023-04-13T18:21:53,110][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded plugin [opensearch-performance-analyzer] jaai-api-opensearch-1 | [2023-04-13T18:21:53,110][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded plugin [opensearch-reports-scheduler] jaai-api-opensearch-1 | [2023-04-13T18:21:53,110][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded plugin [opensearch-security] jaai-api-opensearch-1 | [2023-04-13T18:21:53,110][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded plugin [opensearch-security-analytics] jaai-api-opensearch-1 | [2023-04-13T18:21:53,110][INFO ][o.o.p.PluginsService ] [972572e0721a] loaded plugin [opensearch-sql] jaai-api-opensearch-1 | [2023-04-13T18:21:53,151][DEPRECATION][o.o.d.c.s.Settings ] [972572e0721a] [node.max_local_storage_nodes] setting was deprecated in OpenSearch and will be removed in a future release! See the breaking changes documentation for the next major version. jaai-api-opensearch-1 | [2023-04-13T18:21:53,170][INFO ][o.o.e.NodeEnvironment ] [972572e0721a] using [1] data paths, mounts [[/usr/share/opensearch/data/nodes (grpcfuse)]], net usable_space [262.5gb], net total_space [460.4gb], types [fuse.grpcfuse] jaai-api-opensearch-1 | [2023-04-13T18:21:53,171][INFO ][o.o.e.NodeEnvironment ] [972572e0721a] heap size [1gb], compressed ordinary object pointers [true] jaai-api-api-1 | jaai-api-api-1 | > [email protected] start-local jaai-api-api-1 | > node ./index.js jaai-api-api-1 | jaai-api-opensearch-1 | [2023-04-13T18:21:53,307][INFO ][o.o.n.Node ] [972572e0721a] node name [972572e0721a], node ID [3kh-y0oWTR63NyOHNXTTZw], cluster name [docker-cluster], roles [ingest, remote_cluster_client, data, cluster_manager] jaai-api-api-1 | node:internal/modules/cjs/loader:1078 jaai-api-api-1 | throw err; jaai-api-api-1 | ^ jaai-api-api-1 | jaai-api-api-1 | Error: Cannot find module '@opensearch-project/opensearch' jaai-api-api-1 | Require stack: jaai-api-api-1 | - /app/infrastructure/dependency_injection.js jaai-api-api-1 | - /app/index.js jaai-api-api-1 | at Module._resolveFilename (node:internal/modules/cjs/loader:1075:15) jaai-api-api-1 | at Module._load (node:internal/modules/cjs/loader:920:27) jaai-api-api-1 | at Module.require (node:internal/modules/cjs/loader:1141:19) jaai-api-api-1 | at require (node:internal/modules/cjs/helpers:110:18) jaai-api-api-1 | at Object.<anonymous> (/app/infrastructure/dependency_injection.js:1:20) jaai-api-api-1 | at Module._compile (node:internal/modules/cjs/loader:1254:14) jaai-api-api-1 | at Module._extensions..js (node:internal/modules/cjs/loader:1308:10) jaai-api-api-1 | at Module.load (node:internal/modules/cjs/loader:1117:32) jaai-api-api-1 | at Module._load (node:internal/modules/cjs/loader:958:12) jaai-api-api-1 | at Module.require (node:internal/modules/cjs/loader:1141:19) { jaai-api-api-1 | code: 'MODULE_NOT_FOUND', jaai-api-api-1 | requireStack: [ '/app/infrastructure/dependency_injection.js', '/app/index.js' ] jaai-api-api-1 | } jaai-api-api-1 | jaai-api-api-1 | Node.js v18.15.0 jaai-api-api-1 exited with code 1 jaai-api-opensearch-1 | [2023-04-13T18:21:55,078][INFO ][o.o.a.b.ADCircuitBreakerService] [972572e0721a] Registered memory breaker. jaai-api-opensearch-1 | [2023-04-13T18:21:55,276][INFO ][o.o.m.b.MLCircuitBreakerService] [972572e0721a] Registered ML memory breaker. jaai-api-opensearch-1 | [2023-04-13T18:21:55,276][INFO ][o.o.m.b.MLCircuitBreakerService] [972572e0721a] Registered ML disk breaker. jaai-api-opensearch-1 | [2023-04-13T18:21:55,276][INFO ][o.o.m.b.MLCircuitBreakerService] [972572e0721a] Registered ML native memory breaker. jaai-api-opensearch-1 | [2023-04-13T18:21:55,325][INFO ][o.r.Reflections ] [972572e0721a] Reflections took 19 ms to scan 1 urls, producing 13 keys and 33 values jaai-api-opensearch-1 | [2023-04-13T18:21:55,862][INFO ][o.o.t.NettyAllocator ] [972572e0721a] creating NettyAllocator with the following configs: [name=unpooled, suggested_max_allocation_size=256kb, factors={opensearch.unsafe.use_unpooled_allocator=null, g1gc_enabled=true, g1gc_region_size=1mb, heap_size=1gb}] jaai-api-opensearch-1 | [2023-04-13T18:21:55,934][INFO ][o.o.d.DiscoveryModule ] [972572e0721a] using discovery type [single-node] and seed hosts providers [settings] jaai-api-opensearch-1 | [2023-04-13T18:21:56,199][WARN ][o.o.g.DanglingIndicesState] [972572e0721a] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually jaai-api-opensearch-1 | [2023-04-13T18:21:56,437][INFO ][o.o.p.h.c.PerformanceAnalyzerConfigAction] [972572e0721a] PerformanceAnalyzer Enabled: false jaai-api-opensearch-1 | [2023-04-13T18:21:56,456][INFO ][o.o.n.Node ] [972572e0721a] initialized jaai-api-opensearch-1 | [2023-04-13T18:21:56,456][INFO ][o.o.n.Node ] [972572e0721a] starting ... jaai-api-opensearch-1 | [2023-04-13T18:21:56,521][INFO ][o.o.t.TransportService ] [972572e0721a] publish_address {172.21.0.3:9300}, bound_addresses {0.0.0.0:9300} jaai-api-opensearch-1 | [2023-04-13T18:21:56,699][INFO ][o.o.c.c.Coordinator ] [972572e0721a] cluster UUID [sdwV2ZN4RuKefHdDrlOhbQ] jaai-api-opensearch-1 | [2023-04-13T18:21:56,825][INFO ][o.o.c.s.MasterService ] [972572e0721a] elected-as-cluster-manager ([1] nodes joined)[{972572e0721a}{3kh-y0oWTR63NyOHNXTTZw}{xgDDyRZSQR2EZPUCha7ovg}{172.21.0.3}{172.21.0.3:9300}{dimr}{shard_indexing_pressure_enabled=true} elect leader, _BECOME_CLUSTER_MANAGER_TASK_, _FINISH_ELECTION_], term: 2, version: 8, delta: cluster-manager node changed {previous [], current [{972572e0721a}{3kh-y0oWTR63NyOHNXTTZw}{xgDDyRZSQR2EZPUCha7ovg}{172.21.0.3}{172.21.0.3:9300}{dimr}{shard_indexing_pressure_enabled=true}]} jaai-api-opensearch-1 | [2023-04-13T18:21:56,890][INFO ][o.o.c.s.ClusterApplierService] [972572e0721a] cluster-manager node changed {previous [], current [{972572e0721a}{3kh-y0oWTR63NyOHNXTTZw}{xgDDyRZSQR2EZPUCha7ovg}{172.21.0.3}{172.21.0.3:9300}{dimr}{shard_indexing_pressure_enabled=true}]}, term: 2, version: 8, reason: Publication{term=2, version=8} jaai-api-opensearch-1 | [2023-04-13T18:21:56,894][INFO ][o.o.a.c.ADClusterEventListener] [972572e0721a] Cluster is not recovered yet. jaai-api-opensearch-1 | [2023-04-13T18:21:56,898][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [972572e0721a] Detected cluster change event for destination migration jaai-api-opensearch-1 | [2023-04-13T18:21:56,919][INFO ][o.o.i.i.ManagedIndexCoordinator] [972572e0721a] Cache cluster manager node onClusterManager time: 1681410116919 jaai-api-opensearch-1 | [2023-04-13T18:21:56,928][WARN ][o.o.p.c.s.h.ConfigOverridesClusterSettingHandler] [972572e0721a] Config override setting update called with empty string. Ignoring. jaai-api-opensearch-1 | [2023-04-13T18:21:56,933][INFO ][o.o.d.PeerFinder ] [972572e0721a] setting findPeersInterval to [1s] as node commission status = [true] for local node [{972572e0721a}{3kh-y0oWTR63NyOHNXTTZw}{xgDDyRZSQR2EZPUCha7ovg}{172.21.0.3}{172.21.0.3:9300}{dimr}{shard_indexing_pressure_enabled=true}] jaai-api-opensearch-1 | [2023-04-13T18:21:56,939][INFO ][o.o.h.AbstractHttpServerTransport] [972572e0721a] publish_address {172.21.0.3:9200}, bound_addresses {0.0.0.0:9200} jaai-api-opensearch-1 | [2023-04-13T18:21:56,939][INFO ][o.o.n.Node ] [972572e0721a] started jaai-api-opensearch-1 | [2023-04-13T18:21:56,939][INFO ][o.o.s.OpenSearchSecurityPlugin] [972572e0721a] Node started jaai-api-opensearch-1 | [2023-04-13T18:21:56,940][INFO ][o.o.s.OpenSearchSecurityPlugin] [972572e0721a] 0 OpenSearch Security modules loaded so far: [] jaai-api-opensearch-1 | [2023-04-13T18:21:56,962][INFO ][o.o.c.s.ClusterSettings ] [972572e0721a] updating [plugins.index_state_management.template_migration.control] from [0] to [-1] jaai-api-opensearch-1 | [2023-04-13T18:21:56,963][INFO ][o.o.a.c.HashRing ] [972572e0721a] Node added: [3kh-y0oWTR63NyOHNXTTZw] jaai-api-opensearch-1 | [2023-04-13T18:21:56,965][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [972572e0721a] Detected cluster change event for destination migration jaai-api-opensearch-1 | [2023-04-13T18:21:56,966][INFO ][o.o.a.c.HashRing ] [972572e0721a] Add data node to AD version hash ring: 3kh-y0oWTR63NyOHNXTTZw jaai-api-opensearch-1 | [2023-04-13T18:21:56,967][INFO ][o.o.a.c.HashRing ] [972572e0721a] All nodes with known AD version: {3kh-y0oWTR63NyOHNXTTZw=ADNodeInfo{version=2.6.0, isEligibleDataNode=true}} jaai-api-opensearch-1 | [2023-04-13T18:21:56,967][INFO ][o.o.a.c.HashRing ] [972572e0721a] Rebuild AD hash ring for realtime AD with cooldown, nodeChangeEvents size 0 jaai-api-opensearch-1 | [2023-04-13T18:21:56,967][INFO ][o.o.a.c.HashRing ] [972572e0721a] Build AD version hash ring successfully jaai-api-opensearch-1 | [2023-04-13T18:21:56,968][INFO ][o.o.a.c.ADDataMigrator ] [972572e0721a] Start migrating AD data jaai-api-opensearch-1 | [2023-04-13T18:21:56,968][INFO ][o.o.a.c.ADDataMigrator ] [972572e0721a] AD job index doesn't exist, no need to migrate jaai-api-opensearch-1 | [2023-04-13T18:21:56,968][INFO ][o.o.a.c.ADClusterEventListener] [972572e0721a] Init AD version hash ring successfully jaai-api-opensearch-1 | [2023-04-13T18:21:56,982][INFO ][o.o.g.GatewayService ] [972572e0721a] recovered [1] indices into cluster_state jaai-api-opensearch-1 | [2023-04-13T18:21:56,984][WARN ][o.o.o.i.ObservabilityIndex] [972572e0721a] message: index [.opensearch-observability/la0fMvTITwiAP4IdJwNcfQ] already exists jaai-api-opensearch-1 | [2023-04-13T18:21:56,984][INFO ][o.o.o.i.ObservabilityTracesIndex] [972572e0721a] observability:createMappingTemplate sso_trace_template API called jaai-api-opensearch-1 | [2023-04-13T18:21:57,101][INFO ][o.o.p.PluginsService ] [972572e0721a] PluginService:onIndexModule index:[.opensearch-observability/la0fMvTITwiAP4IdJwNcfQ] jaai-api-opensearch-1 | [2023-04-13T18:21:57,210][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [972572e0721a] Detected cluster change event for destination migration jaai-api-opensearch-1 | [2023-04-13T18:21:57,216][INFO ][o.o.p.PluginsService ] [972572e0721a] PluginService:onIndexModule index:[validate-template-n7ieqxomt9sz6dndwhk_la/yK_nvEU_SQCWMgYhtEfD9A] jaai-api-opensearch-1 | [2023-04-13T18:21:57,240][INFO ][o.o.c.m.MetadataIndexTemplateService] [972572e0721a] updating index template [sso_trace_template] for index patterns [sso_traces-*-*] jaai-api-opensearch-1 | [2023-04-13T18:21:57,399][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [972572e0721a] Detected cluster change event for destination migration jaai-api-opensearch-1 | [2023-04-13T18:21:57,400][INFO ][o.o.o.i.ObservabilityTracesIndex] [972572e0721a] observability:Mapping Template sso_trace_template creation Acknowledged jaai-api-opensearch-1 | [2023-04-13T18:21:57,400][INFO ][o.o.o.i.ObservabilityMetricsIndex] [972572e0721a] observability:createMappingTemplate sso_metric_template API called jaai-api-opensearch-1 | [2023-04-13T18:21:57,401][INFO ][o.o.c.r.a.AllocationService] [972572e0721a] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[.opensearch-observability][0]]]). jaai-api-opensearch-1 | [2023-04-13T18:21:57,527][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [972572e0721a] Detected cluster change event for destination migration jaai-api-opensearch-1 | [2023-04-13T18:21:57,530][INFO ][o.o.p.PluginsService ] [972572e0721a] PluginService:onIndexModule index:[validate-template-csd-euf0qyquyzw8kv4xqg/j6EounzOTAGk0XqOiP57Zg] jaai-api-opensearch-1 | [2023-04-13T18:21:57,541][INFO ][o.o.c.m.MetadataIndexTemplateService] [972572e0721a] updating index template [sso_metric_template] for index patterns [sso_metrics-*-*] jaai-api-opensearch-1 | [2023-04-13T18:21:57,637][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [972572e0721a] Detected cluster change event for destination migration jaai-api-opensearch-1 | [2023-04-13T18:21:57,637][INFO ][o.o.o.i.ObservabilityMetricsIndex] [972572e0721a] observability:Mapping Template sso_metric_template creation Acknowledged jaai-api-opensearch-1 | [2023-04-13T18:22:56,934][INFO ][o.o.i.i.ManagedIndexCoordinator] [972572e0721a] Performing move cluster state metadata. jaai-api-opensearch-1 | [2023-04-13T18:22:56,938][INFO ][o.o.i.i.MetadataService ] [972572e0721a] ISM config index not exist, so we cancel the metadata migration job. jaai-api-opensearch-1 | [2023-04-13T18:23:56,939][INFO ][o.o.i.i.ManagedIndexCoordinator] [972572e0721a] Cancel background move metadata process. jaai-api-opensearch-1 | [2023-04-13T18:23:56,954][INFO ][o.o.i.i.ManagedIndexCoordinator] [972572e0721a] Performing move cluster state metadata. jaai-api-opensearch-1 | [2023-04-13T18:23:56,955][INFO ][o.o.i.i.MetadataService ] [972572e0721a] Move metadata has finished.