Giter Site home page Giter Site logo

ambari-redis-service's Introduction

Ambari-Redis-Service

Project

Redis 5.0.x

Feature Description
Version 5.0.0
Service Redis
Component-Master Redis-Master
Component-Slave Redis-Slave

Prerequisite

  1. Internet Access (need to yum install ruby etc);
  2. Ambari 2.x (test on 2.6.2.0) fully installed.

Features

  1. Support redis single instance deployment: 1 instance;
  2. Support redis cluster mode deployment: 3+ master instance and 3+ slave instance.

Installation

  1. Download the source code.
  2. Copy the project folder to ambari 's lib folder. Such as:
cp -r ./REDIS-5.0.x /var/lib/ambari-server/resources/stacks/HDP/2.6/services
  1. Restart ambari-server:
ambari-server restart
  1. Log into web then add service as usual: add redis service

  2. Configure the service installation:

For single instance deployment

1). choose 1 node be the master: config master

2). choose 0 node be the slave: config slave

3). configure down load url: config download url

4). configure cluster disabled: config cluster disabled

For cluster mode deployment

1). choose at least 3 nodes be the master: config master

2). choose at least 3 nodes be the slave: config slave

3). configure down load url: config download url

4). configure cluster enabled: config cluster enabled

5). configure network: config network

  1. Then continue the deployment and enjoy urself.^^

Background Maintaince

Here is some manually maintaince method specially for cluster operations (All the {} pamas can be found in Redis Configs in Ambari Web):

Check cluster:

{redis.base.dir}/src/redis-cli --cluster check {host_ip}:{port}

Create cluster:

{redis.base.dir}/src/redis-cli --cluster create host1_ip:{master-port} host2_ip:{master-port} host3_ip:{master-port} ...

Add master node to cluster:

{redis.base.dir}/src/redis-cli --cluster add-node {new_host_ip}:{master-port} {exist_cluster_host_ip}:{master-port}
{redis.base.dir}/src/redis-cli --cluster reshard {new_host_ip}:{master-port}

Add slave node to cluster:

{redis.base.dir}/src/redis-cli --cluster add-node {new_host_ip}:{master-port} {exist_cluster_host_ip}:{slave-port} --cluster-slave

Remove node from cluster:

# get node_id by check or info 
{redis.base.dir}/src/redis-cli --cluster check {host_ip}:{master-port}

# then remove
{redis.base.dir}/src/redis-cli --cluster del-node {host_ip}:{port} {node_id}

Clean up node:

# if node is already removed from cluster and stopped.
rm -fr {redis.base.dir} && rm -fr {log.file.dir} && rm -fr {pid.file.dir} && rm -fr {dir}

Re add a removed node back to cluster:

# remove the nodes.conf in dir
rm -fr {dir}/nodes.conf

# start redis (can be done on web)
{redis.base.dir}/src/redis-server {redis.base.dir}/redis.conf

# cleanup db
{redis.base.dir}/src/redis-cli
> flushdb

# use method above-mentioned add node to master or slave

Notice

  1. Only support creating master node cluster for the first time.
  2. Master node scaling need to be done mannually.
  3. Support salve node incremental scaling.
  4. Need more test if using on production.

ambari-redis-service's People

Contributors

balabalayi avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar

ambari-redis-service's Issues

HDP 3.1 installation issues and fixes

Hello,
Bellow you can see some issues that I have encountered on HDP 3.1.
I present my quick fixes (I don't know if this is the best approach but maybe will help someone :) )

  1. Error:
    File "/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/shared_initialization.py", line 50, in setup_users
    groups = params.user_to_groups_dict[user],
    KeyError: u'redis'

Fix:
https://community.hortonworks.com/questions/212871/add-custom-service-via-ambari-error.html
I have choose 'python configs.py' solution.

  1. Error:
    ImportError: cannot import name format_hdp_stack_version

Fix:
In file REDIS-5.0.0/package/scripts/params.py
replace 'format_hdp_stack_version' with 'format_stack_version'

  1. Error:
    File "/var/lib/ambari-agent/cache/stacks/HDP/3.1/services/REDIS-5.0.0/package/scripts/redisSlave.py", line 30, in install
    ignore_failures=True
    TypeError: new() takes at least 2 arguments (1 given)

Fix:
Remove 'username' from 'User' object creation in both redisSlave.py and redisMaster.py.
E.g.
User(params.redis_user,
gid=params.redis_group,
groups=[params.redis_group],
ignore_failures=True
)

hdp_3.1_fix.txt

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.