clickhouse / graphouse Goto Github PK
View Code? Open in Web Editor NEWGraphouse allows you to use ClickHouse as a Graphite storage.
License: Apache License 2.0
Graphouse allows you to use ClickHouse as a Graphite storage.
License: Apache License 2.0
I am evaluating the usage of graphouse and I am struggling with the retention. As far as I understand the retention-config you configure a start point for a specific precision but there is no endpoint. So how will old data ever be deleted from the database?
If not, are you thinking that let graphouse support it?
It would be very nice to support the pickle input format. This can be easily implemented using some pickle serializer for Java. Besides, all similar backends support this feature.
Convert graphite functions to ClickHouse queries. Only JSON format for now.
In this scope:
Hello.
I've recognized, that graphouse doesn't save all incoming metrics until his (?) metrics tree won't fill. It looks like MetricCacher does a lot of side logic synchronously and couldn't accept everything. Increasing memory consumption and deep of in-memory tree does decrease the time of warming-up, but the main problem is data losing.
Here's my custom properties:
graphouse.cacher.read-batch-size=1000
graphouse.cacher.min-batch-size=200000
graphouse.tree.in-memory-levels=7
And vmoptions:
-Xms7g
-Xmx7g
-Xss2m
-XX:StringTableSize=10000000
-XX:+UseG1GC
-XX:MaxGCPauseMillis=1000
Metrics statistic: 2019-03-16 13:16:56,786 INFO [MetricSearch MetricSearch thread] Actual metrics count = 2491274, dir count: 653674, cache stats: CacheStats{hitCount=34678346, missCount=5402061, loadSuccessCount=175284, loadExceptionCount=24, totalLoadTime=1749975810283, evictionCount=0}
graphouse.tree.in-memory-levels=6
graphouse.tree.in-memory-levels=7
SELECT
count(),
toStartOfMinute(toDateTime(timestamp)) AS minute
FROM graphite.data
WHERE (timestamp >= toDateTime('2019-03-16 12:03:00')) AND (timestamp <= now())
GROUP BY minute
ORDER BY minute ASC
┌─count()─┬──────────────minute─┐
│ 2148456 │ 2019-03-16 12:03:00 │
│ 2214717 │ 2019-03-16 12:04:00 │
│ 2241569 │ 2019-03-16 12:05:00 │
│ 2206572 │ 2019-03-16 12:06:00 │
│ 2220109 │ 2019-03-16 12:07:00 │
│ 2154098 │ 2019-03-16 12:08:00 │
│ 2195761 │ 2019-03-16 12:09:00 │
│ 228084 │ 2019-03-16 12:10:00 │
│ 280771 │ 2019-03-16 12:11:00 │
│ 344394 │ 2019-03-16 12:12:00 │
│ 441295 │ 2019-03-16 12:13:00 │
│ 390031 │ 2019-03-16 12:14:00 │
│ 510775 │ 2019-03-16 12:15:00 │
│ 612913 │ 2019-03-16 12:16:00 │
│ 703469 │ 2019-03-16 12:17:00 │
│ 1009436 │ 2019-03-16 12:18:00 │
│ 1436698 │ 2019-03-16 12:19:00 │
│ 1325010 │ 2019-03-16 12:20:00 │
│ 1976466 │ 2019-03-16 12:21:00 │
│ 1862134 │ 2019-03-16 12:22:00 │
│ 1914763 │ 2019-03-16 12:23:00 │
│ 1686410 │ 2019-03-16 12:24:00 │
│ 1643832 │ 2019-03-16 12:25:00 │
│ 1623769 │ 2019-03-16 12:26:00 │
│ 1492997 │ 2019-03-16 12:27:00 │
│ 1816918 │ 2019-03-16 12:28:00 │
│ 2060844 │ 2019-03-16 12:29:00 │
│ 2289562 │ 2019-03-16 12:30:00 │
│ 2191460 │ 2019-03-16 12:31:00 │
│ 2202839 │ 2019-03-16 12:32:00 │
│ 2151755 │ 2019-03-16 12:33:00 │
│ 2187679 │ 2019-03-16 12:34:00 │
│ 2222659 │ 2019-03-16 12:35:00 │
│ 2167668 │ 2019-03-16 12:36:00 │
│ 1052516 │ 2019-03-16 12:37:00 │
│ 298195 │ 2019-03-16 12:38:00 │
│ 396183 │ 2019-03-16 12:39:00 │
│ 442054 │ 2019-03-16 12:40:00 │
│ 532048 │ 2019-03-16 12:41:00 │
│ 811981 │ 2019-03-16 12:42:00 │
│ 822505 │ 2019-03-16 12:43:00 │
│ 847533 │ 2019-03-16 12:44:00 │
│ 889592 │ 2019-03-16 12:45:00 │
│ 953051 │ 2019-03-16 12:46:00 │
│ 868839 │ 2019-03-16 12:47:00 │
│ 1192575 │ 2019-03-16 12:48:00 │
│ 935722 │ 2019-03-16 12:49:00 │
│ 1322319 │ 2019-03-16 12:50:00 │
│ 2155971 │ 2019-03-16 12:51:00 │
│ 2188373 │ 2019-03-16 12:52:00 │
│ 2200898 │ 2019-03-16 12:53:00 │
│ 2202605 │ 2019-03-16 12:54:00 │
│ 2267009 │ 2019-03-16 12:55:00 │
│ 2199843 │ 2019-03-16 12:56:00 │
│ 2159995 │ 2019-03-16 12:57:00 │
│ 2195384 │ 2019-03-16 12:58:00 │
│ 2179685 │ 2019-03-16 12:59:00 │
│ 2429822 │ 2019-03-16 13:00:00 │
│ 2196273 │ 2019-03-16 13:01:00 │
│ 2204160 │ 2019-03-16 13:02:00 │
│ 2249884 │ 2019-03-16 13:03:00 │
│ 2210148 │ 2019-03-16 13:04:00 │
│ 2014119 │ 2019-03-16 13:05:00 │
└─────────┴─────────────────────┘
2019-03-16 12:44:41,566 WARN [MetricServer pool-4-thread-71] Failed to read data
com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: ru.yandex.clickhouse.except.ClickHouseUnknownException: ClickHouse exception, message: Connect to aw-graphite-ch.admin.ig.local:8123 [aw-graphite-ch.admin.ig.local/2a00:1f78:fffd:402a:0:0:0:30d, aw-graphite-ch.admin.ig.local/10.42.3.13] failed: connect timed out, host: aw-graphite-ch.admin.ig.local, port: 8123; Connect to aw-graphite-ch.admin.ig.local:8123 [aw-graphite-ch.admin.ig.local/2a00:1f78:fffd:402a:0:0:0:30d, aw-graphite-ch.admin.ig.local/10.42.3.13] failed: connect timed out
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2217) ~[guava-21.0.jar:?]
at com.google.common.cache.LocalCache.get(LocalCache.java:4154) ~[guava-21.0.jar:?]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4158) ~[guava-21.0.jar:?]
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5147) ~[guava-21.0.jar:?]
at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5153) ~[guava-21.0.jar:?]
at ru.yandex.market.graphouse.search.tree.LoadableMetricDir.getContent(LoadableMetricDir.java:22) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.search.tree.LoadableMetricDir.getMetrics(LoadableMetricDir.java:37) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.search.tree.MetricDir.getOrCreateMetric(MetricDir.java:74) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.search.tree.MetricTree.modify(MetricTree.java:237) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.search.tree.MetricTree.add(MetricTree.java:184) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.search.MetricSearch.add(MetricSearch.java:458) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.server.MetricFactory.createMetric(MetricFactory.java:66) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.server.MetricServer$MetricServerWorker.read(MetricServer.java:117) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.server.MetricServer$MetricServerWorker.run(MetricServer.java:99) [graphouse-1.1-SNAPSHOT.jar:?]
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.base/java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.lang.RuntimeException: ru.yandex.clickhouse.except.ClickHouseUnknownException: ClickHouse exception, message: Connect to aw-graphite-ch.admin.ig.local:8123 [aw-graphite-ch.admin.ig.local/2a00:1f78:fffd:402a:0:0:0:30d, aw-graphite-ch.admin.ig.local/10.42.3.13] failed: connect timed out, host: aw-graphite-ch.admin.ig.local, port: 8123; Connect to aw-graphite-ch.admin.ig.local:8123 [aw-graphite-ch.admin.ig.local/2a00:1f78:fffd:402a:0:0:0:30d, aw-graphite-ch.admin.ig.local/10.42.3.13] failed: connect timed out
at ru.yandex.clickhouse.ClickHouseConnectionImpl.initTimeZone(ClickHouseConnectionImpl.java:94) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseConnectionImpl.<init>(ClickHouseConnectionImpl.java:78) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseDriver.connect(ClickHouseDriver.java:55) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseDataSource.getConnection(ClickHouseDataSource.java:44) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseDataSource.getConnection(ClickHouseDataSource.java:15) ~[clickhouse-jdbc-0.1.50.jar:?]
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:630) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:695) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:727) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:752) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:767) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at ru.yandex.market.graphouse.search.MetricSearch.loadDirsContent(MetricSearch.java:220) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.search.tree.DirContentBatcher.loadDirContent(DirContentBatcher.java:56) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.search.MetricSearch$1.load(MetricSearch.java:159) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.search.MetricSearch$1.load(MetricSearch.java:156) ~[graphouse-1.1-SNAPSHOT.jar:?]
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3716) ~[guava-21.0.jar:?]
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2424) ~[guava-21.0.jar:?]
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2298) ~[guava-21.0.jar:?]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2211) ~[guava-21.0.jar:?]
... 18 more
Caused by: ru.yandex.clickhouse.except.ClickHouseUnknownException: ClickHouse exception, message: Connect to aw-graphite-ch.admin.ig.local:8123 [aw-graphite-ch.admin.ig.local/2a00:1f78:fffd:402a:0:0:0:30d, aw-graphite-ch.admin.ig.local/10.42.3.13] failed: connect timed out, host: aw-graphite-ch.admin.ig.local, port: 8123; Connect to aw-graphite-ch.admin.ig.local:8123 [aw-graphite-ch.admin.ig.local/2a00:1f78:fffd:402a:0:0:0:30d, aw-graphite-ch.admin.ig.local/10.42.3.13] failed: connect timed out
at ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.specify(ClickHouseExceptionSpecifier.java:62) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.specify(ClickHouseExceptionSpecifier.java:24) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseStatementImpl.getInputStream(ClickHouseStatementImpl.java:633) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:127) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:110) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:105) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:100) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseConnectionImpl.initTimeZone(ClickHouseConnectionImpl.java:89) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseConnectionImpl.<init>(ClickHouseConnectionImpl.java:78) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseDriver.connect(ClickHouseDriver.java:55) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseDataSource.getConnection(ClickHouseDataSource.java:44) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseDataSource.getConnection(ClickHouseDataSource.java:15) ~[clickhouse-jdbc-0.1.50.jar:?]
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:630) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:695) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:727) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:752) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:767) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at ru.yandex.market.graphouse.search.MetricSearch.loadDirsContent(MetricSearch.java:220) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.search.tree.DirContentBatcher.loadDirContent(DirContentBatcher.java:56) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.search.MetricSearch$1.load(MetricSearch.java:159) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.search.MetricSearch$1.load(MetricSearch.java:156) ~[graphouse-1.1-SNAPSHOT.jar:?]
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3716) ~[guava-21.0.jar:?]
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2424) ~[guava-21.0.jar:?]
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2298) ~[guava-21.0.jar:?]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2211) ~[guava-21.0.jar:?]
... 18 more
Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to aw-graphite-ch.admin.ig.local:8123 [aw-graphite-ch.admin.ig.local/2a00:1f78:fffd:402a:0:0:0:30d, aw-graphite-ch.admin.ig.local/10.42.3.13] failed: connect timed out
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:150) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) ~[httpclient-4.5.2.jar:4.5.2]
at ru.yandex.clickhouse.ClickHouseStatementImpl.getInputStream(ClickHouseStatementImpl.java:614) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:127) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:110) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:105) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:100) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseConnectionImpl.initTimeZone(ClickHouseConnectionImpl.java:89) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseConnectionImpl.<init>(ClickHouseConnectionImpl.java:78) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseDriver.connect(ClickHouseDriver.java:55) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseDataSource.getConnection(ClickHouseDataSource.java:44) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseDataSource.getConnection(ClickHouseDataSource.java:15) ~[clickhouse-jdbc-0.1.50.jar:?]
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:630) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:695) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:727) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:752) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:767) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at ru.yandex.market.graphouse.search.MetricSearch.loadDirsContent(MetricSearch.java:220) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.search.tree.DirContentBatcher.loadDirContent(DirContentBatcher.java:56) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.search.MetricSearch$1.load(MetricSearch.java:159) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.search.MetricSearch$1.load(MetricSearch.java:156) ~[graphouse-1.1-SNAPSHOT.jar:?]
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3716) ~[guava-21.0.jar:?]
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2424) ~[guava-21.0.jar:?]
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2298) ~[guava-21.0.jar:?]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2211) ~[guava-21.0.jar:?]
... 18 more
Caused by: java.net.SocketTimeoutException: connect timed out
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:?]
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399) ~[?:?]
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242) ~[?:?]
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224) ~[?:?]
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403) ~[?:?]
at java.base/java.net.Socket.connect(Socket.java:591) ~[?:?]
at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:74) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:141) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) ~[httpclient-4.5.2.jar:4.5.2]
at ru.yandex.clickhouse.ClickHouseStatementImpl.getInputStream(ClickHouseStatementImpl.java:614) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:127) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:110) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:105) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:100) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseConnectionImpl.initTimeZone(ClickHouseConnectionImpl.java:89) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseConnectionImpl.<init>(ClickHouseConnectionImpl.java:78) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseDriver.connect(ClickHouseDriver.java:55) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseDataSource.getConnection(ClickHouseDataSource.java:44) ~[clickhouse-jdbc-0.1.50.jar:?]
at ru.yandex.clickhouse.ClickHouseDataSource.getConnection(ClickHouseDataSource.java:15) ~[clickhouse-jdbc-0.1.50.jar:?]
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:630) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:695) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:727) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:752) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:767) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at ru.yandex.market.graphouse.search.MetricSearch.loadDirsContent(MetricSearch.java:220) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.search.tree.DirContentBatcher.loadDirContent(DirContentBatcher.java:56) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.search.MetricSearch$1.load(MetricSearch.java:159) ~[graphouse-1.1-SNAPSHOT.jar:?]
at ru.yandex.market.graphouse.search.MetricSearch$1.load(MetricSearch.java:156) ~[graphouse-1.1-SNAPSHOT.jar:?]
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3716) ~[guava-21.0.jar:?]
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2424) ~[guava-21.0.jar:?]
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2298) ~[guava-21.0.jar:?]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2211) ~[guava-21.0.jar:?]
... 18 more
This is side problem: ru.yandex.market.graphouse.search.tree.DirContentBatcher.loadDirContent always executes this code and doesn't create batches. This overload ClickHouse a lot and cause DB side lags.
New version of graphite-web is now supporting python 3.
I ve tried to setup everything and i was unable to display any data.
After looking in the driver ( and spending some time , i'm not a python dev , so apology if the vocabulary is not correct ) , i 've found 2 blocking issue in graphouse.py
1: in the init method of the graphouseReader, the way to detect if we have a collection or a string is not working ( hasattr(path, 'iter') is returning true on string , so it breaks everything, using something like isinstance(obj, collections.Sequence) should solve the issue )
2 : The fetch method is now taking 5 parameters instead of 3 ( self, start,stop,now,requestContext).
By fixing those 2 issues , i was able to make things work. But as i ve starting to learn python during the troubleshooting i don't feel really confortable to propose a fix.
Regards
OS: Ubuntu 16.04.5
Arch: amd64
Java: 1.8.0_181 (Oracle implementation)
RAM: 6GB
Graphouse/Clickhouse installed according to manual https://github.com/yandex/graphouse/blob/master/doc/install.md
I am trying to start Graphouse but seeing error:
root@ubuntu:/var/log/graphouse# USER="graphouse"
root@ubuntu:/var/log/graphouse# SERVICE="graphouse"
root@ubuntu:/var/log/graphouse# GRAPHOUSE_ROOT=/opt/graphouse
root@ubuntu:/var/log/graphouse# SHELL_LOG=/opt/graphouse/log/graphouse.shell.log
root@ubuntu:/var/log/graphouse# PID_FILE=/var/run/$SERVICE.pid
root@ubuntu:/var/log/graphouse# /sbin/start-stop-daemon --start --exec $GRAPHOUSE_ROOT/bin/graphouse --make-pidfile --pidfile $PID_FILE --background --no-close --chuid $USER
root@ubuntu:/var/log/graphouse# java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)
Graphouse path is: /opt/graphouse
Sep 25, 2018 11:57:55 AM org.springframework.context.annotation.AnnotationConfigApplicationContext prepareRefresh
INFO: Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@1d16f93d: startup date [Tue Sep 25 11:57:55 EDT 2018]; root of context hierarchy
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Sep 25, 2018 11:57:56 AM org.springframework.context.annotation.AnnotationConfigApplicationContext refresh
WARNING: Exception encountered during context initialization - cancelling refresh attempt
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'graphouseConfig': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private ru.yandex.market.graphouse.search.MetricSearch ru.yandex.market.graphouse.config.GraphouseConfig.metricSearch; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ru.yandex.market.graphouse.config.MetricsConfig': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private org.springframework.jdbc.core.JdbcTemplate ru.yandex.market.graphouse.config.MetricsConfig.clickHouseJdbcTemplate; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'clickHouseJdbcTemplate' defined in class ru.yandex.market.graphouse.config.DbConfig: Unsatisfied dependency expressed through constructor argument with index 0 of type [javax.sql.DataSource]: : Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:334)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1210)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:755)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:757)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:480)
at ru.yandex.market.graphouse.GraphouseMain.main(GraphouseMain.java:27)
Caused by: org.springframework.beans.factory.BeanCreationException: Could not autowire field: private ru.yandex.market.graphouse.search.MetricSearch ru.yandex.market.graphouse.config.GraphouseConfig.metricSearch; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ru.yandex.market.graphouse.config.MetricsConfig': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private org.springframework.jdbc.core.JdbcTemplate ru.yandex.market.graphouse.config.MetricsConfig.clickHouseJdbcTemplate; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'clickHouseJdbcTemplate' defined in class ru.yandex.market.graphouse.config.DbConfig: Unsatisfied dependency expressed through constructor argument with index 0 of type [javax.sql.DataSource]: : Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:561)
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:331)
... 11 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ru.yandex.market.graphouse.config.MetricsConfig': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private org.springframework.jdbc.core.JdbcTemplate ru.yandex.market.graphouse.config.MetricsConfig.clickHouseJdbcTemplate; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'clickHouseJdbcTemplate' defined in class ru.yandex.market.graphouse.config.DbConfig: Unsatisfied dependency expressed through constructor argument with index 0 of type [javax.sql.DataSource]: : Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:334)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1210)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:368)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1119)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1014)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1120)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1044)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:942)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:533)
... 13 more
Caused by: org.springframework.beans.factory.BeanCreationException: Could not autowire field: private org.springframework.jdbc.core.JdbcTemplate ru.yandex.market.graphouse.config.MetricsConfig.clickHouseJdbcTemplate; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'clickHouseJdbcTemplate' defined in class ru.yandex.market.graphouse.config.DbConfig: Unsatisfied dependency expressed through constructor argument with index 0 of type [javax.sql.DataSource]: : Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:561)
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:331)
... 33 more
Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'clickHouseJdbcTemplate' defined in class ru.yandex.market.graphouse.config.DbConfig: Unsatisfied dependency expressed through constructor argument with index 0 of type [javax.sql.DataSource]: : Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:749)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:464)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1119)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1014)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1120)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1044)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:942)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:533)
... 35 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:599)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1119)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1014)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1120)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1044)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:942)
at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:813)
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:741)
... 48 more
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:189)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:588)
... 61 more
Caused by: java.lang.IllegalArgumentException: Incorrect url
at ru.yandex.clickhouse.BalancedClickhouseDataSource.splitUrl(BalancedClickhouseDataSource.java:121)
at ru.yandex.clickhouse.BalancedClickhouseDataSource.<init>(BalancedClickhouseDataSource.java:71)
at ru.yandex.market.graphouse.config.DbConfig.clickHouseDataSource(DbConfig.java:58)
at ru.yandex.market.graphouse.config.DbConfig$$EnhancerBySpringCGLIB$$40992899.CGLIB$clickHouseDataSource$3(<generated>)
at ru.yandex.market.graphouse.config.DbConfig$$EnhancerBySpringCGLIB$$40992899$$FastClassBySpringCGLIB$$b5f530f7.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:309)
at ru.yandex.market.graphouse.config.DbConfig$$EnhancerBySpringCGLIB$$40992899.clickHouseDataSource(<generated>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162)
... 62 more
Exception in thread "main" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'graphouseConfig': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private ru.yandex.market.graphouse.search.MetricSearch ru.yandex.market.graphouse.config.GraphouseConfig.metricSearch; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ru.yandex.market.graphouse.config.MetricsConfig': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private org.springframework.jdbc.core.JdbcTemplate ru.yandex.market.graphouse.config.MetricsConfig.clickHouseJdbcTemplate; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'clickHouseJdbcTemplate' defined in class ru.yandex.market.graphouse.config.DbConfig: Unsatisfied dependency expressed through constructor argument with index 0 of type [javax.sql.DataSource]: : Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:334)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1210)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:755)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:757)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:480)
at ru.yandex.market.graphouse.GraphouseMain.main(GraphouseMain.java:27)
Caused by: org.springframework.beans.factory.BeanCreationException: Could not autowire field: private ru.yandex.market.graphouse.search.MetricSearch ru.yandex.market.graphouse.config.GraphouseConfig.metricSearch; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ru.yandex.market.graphouse.config.MetricsConfig': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private org.springframework.jdbc.core.JdbcTemplate ru.yandex.market.graphouse.config.MetricsConfig.clickHouseJdbcTemplate; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'clickHouseJdbcTemplate' defined in class ru.yandex.market.graphouse.config.DbConfig: Unsatisfied dependency expressed through constructor argument with index 0 of type [javax.sql.DataSource]: : Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:561)
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:331)
... 11 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ru.yandex.market.graphouse.config.MetricsConfig': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private org.springframework.jdbc.core.JdbcTemplate ru.yandex.market.graphouse.config.MetricsConfig.clickHouseJdbcTemplate; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'clickHouseJdbcTemplate' defined in class ru.yandex.market.graphouse.config.DbConfig: Unsatisfied dependency expressed through constructor argument with index 0 of type [javax.sql.DataSource]: : Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:334)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1210)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:368)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1119)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1014)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1120)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1044)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:942)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:533)
... 13 more
Caused by: org.springframework.beans.factory.BeanCreationException: Could not autowire field: private org.springframework.jdbc.core.JdbcTemplate ru.yandex.market.graphouse.config.MetricsConfig.clickHouseJdbcTemplate; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'clickHouseJdbcTemplate' defined in class ru.yandex.market.graphouse.config.DbConfig: Unsatisfied dependency expressed through constructor argument with index 0 of type [javax.sql.DataSource]: : Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:561)
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:331)
... 33 more
Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'clickHouseJdbcTemplate' defined in class ru.yandex.market.graphouse.config.DbConfig: Unsatisfied dependency expressed through constructor argument with index 0 of type [javax.sql.DataSource]: : Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:749)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:464)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1119)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1014)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1120)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1044)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:942)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:533)
... 35 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'clickHouseDataSource' defined in class ru.yandex.market.graphouse.config.DbConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:599)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1119)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1014)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1120)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1044)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:942)
at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:813)
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:741)
... 48 more
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'clickHouseDataSource' threw exception; nested exception is java.lang.IllegalArgumentException: Incorrect url
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:189)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:588)
... 61 more
Caused by: java.lang.IllegalArgumentException: Incorrect url
at ru.yandex.clickhouse.BalancedClickhouseDataSource.splitUrl(BalancedClickhouseDataSource.java:121)
at ru.yandex.clickhouse.BalancedClickhouseDataSource.<init>(BalancedClickhouseDataSource.java:71)
at ru.yandex.market.graphouse.config.DbConfig.clickHouseDataSource(DbConfig.java:58)
at ru.yandex.market.graphouse.config.DbConfig$$EnhancerBySpringCGLIB$$40992899.CGLIB$clickHouseDataSource$3(<generated>)
at ru.yandex.market.graphouse.config.DbConfig$$EnhancerBySpringCGLIB$$40992899$$FastClassBySpringCGLIB$$b5f530f7.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:309)
at ru.yandex.market.graphouse.config.DbConfig$$EnhancerBySpringCGLIB$$40992899.clickHouseDataSource(<generated>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162)
... 62 more
Java HotSpot(TM) 64-Bit Server VM (25.181-b13) for linux-amd64 JRE (1.8.0_181-b13), built on Jul 7 2018 00:56:38 by "java_re" with gcc 4.3.0 20080428 (Red Hat 4.3.0-8)
Memory: 4k page, physical 5965316k(5623184k free), swap 998396k(998396k free)
CommandLine flags: -XX:GCLogFileSize=52428800 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/graphouse/log/ -XX:InitialHeapSize=95445056 -XX:MaxHeapSize=1527120896 -XX:NumberOfGCLogFiles=7 -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseGCLogFileRotation -XX:+UseParallelGC
2018-09-25T11:57:55.856-0400: 0.499: [GC (Allocation Failure) [PSYoungGen: 24064K->3576K(27648K)] 24064K->5447K(90624K), 0.0056892 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
Heap
PSYoungGen total 27648K, used 23327K [0x00000000e1a00000, 0x00000000e5000000, 0x0000000100000000)
eden space 24064K, 82% used [0x00000000e1a00000,0x00000000e2d49c60,0x00000000e3180000)
from space 3584K, 99% used [0x00000000e3180000,0x00000000e34fe208,0x00000000e3500000)
to space 3584K, 0% used [0x00000000e4c80000,0x00000000e4c80000,0x00000000e5000000)
ParOldGen total 62976K, used 1871K [0x00000000a4e00000, 0x00000000a8b80000, 0x00000000e1a00000)
object space 62976K, 2% used [0x00000000a4e00000,0x00000000a4fd3c08,0x00000000a8b80000)
Metaspace used 14817K, capacity 15088K, committed 15232K, reserved 1062912K
class space used 1811K, capacity 1900K, committed 1920K, reserved 1048576K
What am I doing wrong?
Since v1.1.1 graphite could tag metrics.
It could be great to have it also in graphouse.
Guys,
Do you have any plans to use BalancedClickhouseDataSource Thus if telemetry data lives in Distributed, Replicated and Sharded Cluster the load will get spread evenly and guarantee HA if one of CH servers is down in a cluster.
I'm trying to configure graphite-web to read data but I got an exception
Internal Server Error: /graphite/render/ Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/exception.py", line 41, in inner response = get_response(request) File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/opt/graphite/webapp/graphite/render/views.py", line 121, in renderView data.extend(evaluateTarget(requestContext, targets)) File "/opt/graphite/webapp/graphite/render/evaluator.py", line 14, in evaluateTarget prefetchData(requestContext, pathExpressions) File "/opt/graphite/webapp/graphite/render/datalib.py", line 255, in prefetchData for result in STORE.fetch(pathExpressions, startTime, endTime, now, requestContext): File "/opt/graphite/webapp/graphite/storage.py", line 100, in fetch raise Exception('All fetches failed for %s' % (str(patterns))) Exception: All fetches failed for [u'local.random.diceroll']
Hi, we are a bit new to grafite... since we are using ClickHouse we also decided to use graphouse for telemetry. So we are using Grafana with ClickHouse datasource, and going directly to graphouse.data to get telemetry and what we are seeing data is actually 1 min to 5 min lag depending on volume and frequency. So I suspect is graphouse catches some data in memory using the options below (in #Mretric search and tree.. section )... if this is true what is the best setting for pushing metrics to ClickHouse no more than 30 seconds old? Is this possible? What are our options? Any advice is appreciated
We are running on 2 CPU / 8 GB RAM server
graphouse.properties
graphouse.allow-cold-run=false
#Clickhouse
graphouse.clickhouse.host=somehost
graphouse.clickhouse.port=8123
graphouse.clickhouse.db=graphite
graphouse.clickhouse.user=default
graphouse.clickhouse.password=ourpass
graphouse.clickhouse.compress=false
graphouse.clickhouse.data-table=data
graphouse.clickhouse.metric-tree-table=metrics
graphouse.clickhouse.socket-timeout-seconds=600
graphouse.clickhouse.query-timeout-seconds=120
graphouse.clickhouse.retention-config=
#metric server and cacher
graphouse.cacher.bind-address=0.0.0.0
graphouse.cacher.port=2003
graphouse.cacher.threads=2800
graphouse.cacher.socket-timeout-millis=42000
graphouse.cacher.queue-size=10000000
graphouse.cacher.read-batch-size=1000
graphouse.cacher.min-batch-size=10000
graphouse.cacher.max-batch-size=1000000
graphouse.cacher.min-batch-time-seconds=1
graphouse.cacher.max-batch-time-seconds=5
graphouse.cacher.max-output-threads=10000
#Http server (metric search, ping, metricData)
graphouse.http.bind-address=
graphouse.http.port=2005
graphouse.http.threads=25
graphouse.http.max-form-context-size-bytes=500000
#Mretric search and tree
graphouse.search.refresh-seconds=1
graphouse.tree.in-memory-levels=3
graphouse.tree.dir-content.cache-time-minutes=10
graphouse.tree.dir-content.cache-concurrency-level=100
graphouse.tree.dir-content.batcher.max-parallel-requests=3
graphouse.tree.dir-content.batcher.max-batch-size=2000
graphouse.tree.dir-content.batcher.aggregation-time-millis=50
#Host metrics redirect
graphouse.host-metric-redirect.enabled=false
graphouse.host-metric-redirect.dir=
graphouse.host-metric-redirect.postfixes=
#Autohide
graphouse.autohide.enabled=false
graphouse.autohide.run-delay-minutes=30
graphouse.autohide.max-values-count=200
graphouse.autohide.missing-days=7
graphouse.autohide.step=10000
graphouse.autohide.retry.count=10
graphouse.autohide.retry.wait_seconds=10
graphouse.autohide.clickhouse.query-timeout-seconds=600
#Metric validation
graphouse.metric-validation.min-length=5
graphouse.metric-validation.max-length=500
graphouse.metric-validation.min-levels=2
graphouse.metric-validation.max-levels=20
#graphouse.metric-validation.regexp=[-_0-9a-zA-Z\\.]*$
graphouse.metric-validation.regexp=.*
Graphouse не запускался в docker
после обновления.
Лог в консоли:
java version "1.8.0_151"
Java(TM) SE Runtime Environment (build 1.8.0_151-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.151-b12, mixed mode)
Graphouse path is: /opt/graphouse
Dec 22, 2017 2:35:30 PM org.springframework.context.annotation.AnnotationConfigApplicationContext prepareRefresh
INFO: Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@3a5ed7a6: startup date [Fri Dec 22 14:35:30 MSK 2017]; root of context hierarchy
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Dec 22, 2017 2:35:32 PM org.springframework.context.annotation.AnnotationConfigApplicationContext refresh
WARNING: Exception encountered during context initialization - cancelling refresh attempt
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'graphouseWebServer' defined in class ru.yandex.market.graphouse.config.ServerConfig: Invocation of init method failed; nested exception is java.net.SocketException: Protocol family unavailable
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1574)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:539)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:755)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:757)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:480)
at ru.yandex.market.graphouse.GraphouseMain.main(GraphouseMain.java:27)
Caused by: java.net.SocketException: Protocol family unavailable
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:366)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at ru.yandex.market.graphouse.GraphouseWebServer.startServer(GraphouseWebServer.java:82)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1702)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1641)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1570)
... 10 more
Exception in thread "main" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'graphouseWebServer' defined in class ru.yandex.market.graphouse.config.ServerConfig: Invocation of init method failed; nested exception is java.net.SocketException: Protocol family unavailable
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1574)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:539)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:755)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:757)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:480)
at ru.yandex.market.graphouse.GraphouseMain.main(GraphouseMain.java:27)
Caused by: java.net.SocketException: Protocol family unavailable
Проблема решилась следующими параметрами окружения:
- GH__CACHER__BIND_ADDRESS=0.0.0.0
- GH__HTTP__BIND_ADDRESS=0.0.0.0
Или настройками:
graphouse.cacher.bind-address=0.0.0.0
graphouse.http.bind-address=0.0.0.0
Используем связку graphite-web(1.0.2) + graphouse(1.3) + grafana.
При попытке получить данные по нескольким серверам за определенный промежуток времени, если данных по одному из серверов за это время нет, grafana выдает ошибку:
<center> <h2> <p>Graphite encountered an unexpected error while handling your request.</p> <p>Please contact your site administrator if the problem persists.</p> </h2> <br> <div> </div> <div> <pre>Traceback (most recent call last): File "/opt/graphite/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 149, in get_response response = self.process_exception_by_middleware(e, request) File "/opt/graphite/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 147, in get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/opt/graphite/local/lib/python2.7/site-packages/graphite/render/views.py", line 174, in renderView del series[0] IndexError: list assignment index out of range </pre> </div> </center>
При этом, если попытаться получить данные за период, в течении которого данные по всем серверам есть, хотя бы частично то ошибки не возникает. Например, если данные начали появляться сегодня, то график за последние 3 дня ошибки не выдаст. Однако, график за "вчера" построить не удастся.
Если запросить данные из graphouse за период, пересекающийся с периодом когда данные начали поступать в graphouse:
curl 'http://localhost:2005/metricData?metrics=collectd.srv6.nstat-IpOutNoRoutes.gauge&start=1502020061&reqKey=empty&end=1502117769'
Ответ будет следующим:
{"collectd.srv6.nstat-IpOutNoRoutes.gauge":{"start":1502020020,"end":1502117700,"step":60,"points":[null,null,null,null,null,null,null,null,null,null,null,null,null,...,null,null,null,null,null,null,70.0,70.0,...,70.0]}}
Если же уменьшить время в поле end
, то graphouse возвращает пустое значение:
curl 'http://localhost:2005/metricData?metrics=collectd.srv6.nstat-IpOutNoRoutes.gauge&start=1502020061&reqKey=empty&end=1502031061' {}
Если график строить в graphite-web, то он перестает отображать графики как только в выборку попадает сервер, данные с которого за весь указанный период отсутствуют.
Any query performance benchmark for graphite with clickhouse as backend ? I've benchmarked it locally, doesn't look good. is it because clickhouse is not designed for processing a large quantity of short queries ?
I think should be corrected and reflect the actual git branch/tag version... Also this is no should be considered snapshot as we a using it... 😄
https://github.com/yandex/graphouse/blob/d368c1150700adce07e5232d4b68eff28c1e3ffb/build.gradle#L18
The packages in https://repo.yandex.ru/graphouse/trusty/pool/main/g/graphouse/ are quite old.
It would be good to have them updated automatically for new releases.
I'm just finishing up migrating from carbon to graphouse/clickhouse and found that we have a bunch of snmp metrics that failed to backfill/load. I can work around this by using numeric MIBs but thought that I should add an issue here to help others in case they don't have that workaround.
Something like the following will work on graphite/carbon but fails to ingest into clickhouse/graphouse.
root@admin02:~# echo "test.bash.st-a::ts 42 `date +%s`" | nc localhost 2023
root@admin02:~# echo "test.bash.st-a::ts 42 `date +%s`" | nc localhost 2023
root@admin02:~# echo "test.bash.st-a::ts 42 `date +%s`" | nc localhost 2023 ```
I would like to see a maximum retention, similar to carbon/whisper storage-schemas.conf. Data that exceeds the maximum age should be purged from the database automatically.
I shouldn't be difficult so far.
I see the next way:
$partition_timeframe / week (or may be configurable as well)
times if there are retention borders (could be founded in system.graphite_retentions
) and one time after the borders are goneIn our environment I've implemented the next script:
#!/usr/bin/env python
from datetime import timedelta, datetime
from socket import getfqdn
FQDN = '_'.join(getfqdn().split('.')[0:1])
now = datetime.now()
day = timedelta(days=1)
metric_name = 'servers.{}.software.graphouse.fake_history_{}'
for days_ago in range(365 * 4):
ts = int((now - day * days_ago).timestamp())
for i in range(1, 21):
print(metric_name.format(FQDN, i), 1, ts)
So, it's possible to add a scheduller of the fake metric into graphouse in the same way. 2 config options:
graphouse.py use log.info for DEBUG messages:
https://github.com/yandex/graphouse/blob/f41cc517ac519fd0c2bab259d595894b63268dbf/src/main/pySources/graphouse.py#L78
https://github.com/yandex/graphouse/blob/f41cc517ac519fd0c2bab259d595894b63268dbf/src/main/pySources/graphouse.py#L104
I suppose it must be log.debug.
Also this can be changed to log.error, I believe:
https://github.com/yandex/graphouse/blob/f41cc517ac519fd0c2bab259d595894b63268dbf/src/main/pySources/graphouse.py#L82
repository="deb https://repo.yandex.ru/clickhouse/xenial dists/stable/main/binary-amd64/"
apt-get update
apt-get install -y apt-transport-https
mkdir -p /etc/apt/sources.list.d
echo $repository | tee /etc/apt/sources.list.d/clickhouse.list
apt-get update
apt-get install --allow-unauthenticated -y graphouse=$version
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.