Kusabana is coded by Ruby 2.0, using em-proxy gem, and depending on memcached.
Elasticsearch + Kibana are becoming typical solution for data mining.
However, it is said that they have some performance problems. Why?
I think, there are some problems about behavior of Kibana.
For example, Kibana make request to ES on almost clicking.
Futhermore, Elasticsearch doesn’t have a mechanism of ‘query cache’ but for ‘filter cache’.
These problems are seemed to be resolve by proxy that have simple caching system.
However, The query produced by kibana is variable by time, path or dashboard’s environment.
This make us have to create more functional caching.
Kusabana is a solution implemented for this problem.
Needless to say caching, Kusabana can store log of itself to Elasticsearch.
This will make you able to make configration easier.
git clone https://github.com/voyagegroup/kusabana cd kusabana bundle install
memcached gem, it require
libsasl2-dev or any other similer package.
proxy: host: '0.0.0.0' port: 9292 daemonize: false timeout: 15 # output: 'log/kusabana.log' # pid: 'log/kusabana.pid' es: remote: host: 'localhost' port: 9200 # output: # index: 'kusabana-log-1' # hosts: # - host: 'localhost' # port: 9200 cache: url: 'localhost:11211'
If you want to store Kusabana’s log and set output Elasticsearch, You should PUT index template to ES.
bundle rake template:create
Additionally, You can use Kibana Dashboard for montoring Kusabana’s log.
bundle rake dashboard:create
Then, the Dashboard is going to be seen at
The configration of cache is available in
# Default settings rules =  search_caching = Kusabana::Rule.new('POST', /^\/.+\/_search(\?.+)?$/, 300) timestamp_modifier = Kusabana::QueryModifier.new(/@timestamp/) do |query| if query.key?('from') && query.key?('to') query['from'] = query['from'] / 100000 * 100000 query['to'] = query['to'] / 100000 * 100000 end query end search_caching.add_modifier(timestamp_modifier) rules << search_caching rules << Kusabana::Rule.new('GET', /^\/_nodes$/, 300) rules << Kusabana::Rule.new('GET', /^\/\S+\/_mapping$/, 300) rules << Kusabana::Rule.new('GET', /^\/\S+\/_aliases(\?.+)?$/, 300)
Kusabana::Rule.new(method, path_pattern, expire)
When Kusabana serve request, it will be checked whether there are any
Rule that is matched by
If match, Kusabana will try parsing query, scanning JSON’s key and executing
$block of matched
The query will be modified by returning of
This behavior is made for ignoring tiny differents between queries(e.g. The filter by range of
QueryModifier is apply only first matched one.
[took when store]*count/expireSO SUITABLE
Today, Kusabana is experimental product yet.
Don’t mind to send patch!
bundle exec rake test)