Wednesday, October 5, 2016

How to: Apache Cassandra cluster running in a single host

This post created based below URL to setup Apache Cassandra Cluster (original article use DSE as example).

 http://www.datastax.com/dev/blog/running-multiple-datastax-enterprise-nodes-in-a-single-host


The resulting configuration will look like this:
  • ·         Single binary tarball installation: we’ll install Apache Cassandra once, and share it across nodes.
  • ·         Multiple node locations: each node will have its own directory hierarchy with configuration files, data, and logs.

Configure the network interfaces

[root@localhost ~]# ifconfig lo:0 127.0.0.2
[root@localhost ~]# ifconfig lo:1 127.0.0.3

[root@localhost ~]# ip addr
1: lo: mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet 127.0.0.2/8 scope host secondary lo:0
    inet 127.0.0.3/8 scope host secondary lo:1
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever







Install apache cassandra

$ tar zxvf apache-cassandra-3.9-bin.tar.gz

Setting up the nodes

$ for i in 1 2 3; do mkdir -p cluster/dc1/node$i/data cluster/dc1/node$i/logs; done
$ mkdir -p cluster/dc1/node1/conf && cp -r $CASSANDRA_HOME/conf cluster/dc1/node1/
$ mkdir -p cluster/dc1/node2/conf && cp -r $CASSANDRA_HOME/conf cluster/dc1/node2/
$ mkdir -p cluster/dc1/node3/conf && cp -r $CASSANDRA_HOME/conf cluster/dc1/node3/

Prepare Node1/2/3 Configurations


==============/home/donghua/cluster/dc1/node1/conf/cassandra.yaml==============
cluster_name: 'SingleServer Cluster'
data_file_directories:
     - /home/donghua/cluster/dc1/node1/data *****
commitlog_directory: /home/donghua/cluster/dc1/node1/data/commitlog *****
saved_caches_directory: /home/donghua/cluster/dc1/node1/data/saved_caches *****
listen_address: 127.0.0.1 *****
rpc_address: 127.0.0.1 *****

==============/home/donghua/cluster/dc1/node1/conf/logback.xml==============
configuration scan="true"
  jmxConfigurator 
  shutdownHook class="ch.qos.logback.core.hook.DelayingShutdownHook"
  property name="cassandra.logdir" value="/home/donghua/cluster/dc1/node1/logs/" *****

==============/home/donghua/cluster/dc1/node1/conf/jvm.options==============
-Xms200M
-Xmx400M

==============/home/donghua/cluster/dc1/node1/conf/cassandra-env.sh==============
# For security reasons, you should not expose this port to the internet.  Firewall it if needed.
JMX_PORT="7199"  ****7299 for node2, 7399 for node3

==============/home/donghua/cluster/dc1/node1/conf/cassandra-rackdc.properties==============
127.0.0.1=DC1:RAC1
127.0.0.2=DC1:RAC1
127.0.0.3=DC1:RAC1

Prepare Node1/2/3 Startup/Shutdown Scripts


==============/home/donghua/.profile_node1==============
export JAVA_HOME=/usr/local/jdk1.8.0_91
export CASSANDRA_HOME=/home/donghua/apache-cassandra-3.9
export PATH=$CASSANDRA_HOME/bin:$JAVA_HOME/bin:$PATH
export CASSANDRA_CONF=/home/donghua/cluster/dc1/node1/conf


==============/home/donghua/cluster/start_node1.sh==============
#!/bin/bash

export JAVA_HOME=/usr/local/jdk1.8.0_91
export CASSANDRA_HOME=/home/donghua/apache-cassandra-3.9
export PATH=$CASSANDRA_HOME/bin:$JAVA_HOME/bin:$PATH
export CASSANDRA_CONF=/home/donghua/cluster/dc1/node1/conf

$CASSANDRA_HOME/bin/cassandra -p ~/cluster/node1.pid

==============/home/donghua/cluster/stop_node1.sh==============
kill `cat ~/cluster/node1.pid`



Startup Nodes one by one


/home/donghua/cluster/start_node1.sh

[donghua@localhost cluster]$ nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address    Load       Tokens       Owns (effective)  Host ID                               Rack
UN  127.0.0.1  140.18 KiB  256          100.0%            47100081-8400-4996-8dd9-84d66280a1c6  rack1

/home/donghua/cluster/start_node2.sh

[donghua@localhost bin]$ nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address    Load       Tokens       Owns (effective)  Host ID                               Rack
UN  127.0.0.1  140.18 KiB  256          100.0%            47100081-8400-4996-8dd9-84d66280a1c6  rack1
UN  127.0.0.2  108.22 KiB  256          100.0%            e9600f15-e8ca-4a9d-a12a-5617efce4707  rack1


/home/donghua/cluster/start_node3.sh

[donghua@localhost bin]$ nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address    Load       Tokens       Owns (effective)  Host ID                               Rack
UN  127.0.0.1  185.72 KiB  256          69.8%             47100081-8400-4996-8dd9-84d66280a1c6  rack1
UN  127.0.0.2  178.67 KiB  256          65.1%             e9600f15-e8ca-4a9d-a12a-5617efce4707  rack1
UN  127.0.0.3  84.1 KiB   256          65.1%             5a43d82c-6379-4376-b5a6-3c8537efc88a  rack1


Test and verify the cluster


[donghua@localhost cluster]$ cqlsh
Connected to SingleServer Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.9 | CQL spec 3.4.2 | Native protocol v4]
Use HELP for help.
cqlsh> create keyspace mykeyspace WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 3};
cqlsh> use mykeyspace;
cqlsh:mykeyspace> CREATE TABLE t1 (a text,b text,c timestamp,d int,e double,f int,PRIMARY KEY ((a, b), c)) WITH CLUSTERING ORDER BY (c ASC);
cqlsh:mykeyspace> desc t1

CREATE TABLE mykeyspace.t1 (
    a text,
    b text,
    c timestamp,
    d int,
    e double,
    f int,
    PRIMARY KEY ((a, b), c)
) WITH CLUSTERING ORDER BY (c ASC)
    AND bloom_filter_fp_chance = 0.01
    AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
    AND comment = ''
    AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
    AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
    AND crc_check_chance = 1.0
    AND dclocal_read_repair_chance = 0.1
    AND default_time_to_live = 0
    AND gc_grace_seconds = 864000
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND read_repair_chance = 0.0
    AND speculative_retry = '99PERCENTILE';

cqlsh:mykeyspace> COPY t1 FROM '/home/donghua/ts.csv';
Using 1 child processes

Starting copy of mykeyspace.t1 with columns [a, b, c, d, e, f].
Processed: 445674 rows; Rate:     982 rows/s; Avg. rate:    2084 rows/s
445674 rows imported from 1 files in 3 minutes and 33.866 seconds (0 skipped).

[donghua@localhost bin]$ nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address    Load       Tokens       Owns (effective)  Host ID                               Rack
UN  127.0.0.1  9.85 MiB   256          100.0%            47100081-8400-4996-8dd9-84d66280a1c6  rack1
UN  127.0.0.2  7.28 MiB   256          100.0%            e9600f15-e8ca-4a9d-a12a-5617efce4707  rack1
UN  127.0.0.3  9.5 MiB    256          100.0%            5a43d82c-6379-4376-b5a6-3c8537efc88a  rack1

[donghua@localhost dc1]$ cqlsh 127.0.0.1 9042 -e "select * from mykeyspace.t1 limit 3"

 a      | b        | c                               | d | e         | f
--------+----------+---------------------------------+---+-----------+------
 100003 | 20161005 | 2016-10-05 10:51:29.157000+0000 | 2 | 115.40718 | 3035
 100003 | 20161005 | 2016-10-05 10:51:29.906000+0000 | 6 |  142.8354 |  579
 100003 | 20161005 | 2016-10-05 10:51:32.918000+0000 | 2 |   9.87186 | 6030

(3 rows)
[donghua@localhost dc1]$ cqlsh 127.0.0.2 9042 -e "select * from mykeyspace.t1 limit 3"

 a      | b        | c                               | d | e         | f
--------+----------+---------------------------------+---+-----------+------
 100003 | 20161005 | 2016-10-05 10:51:29.157000+0000 | 2 | 115.40718 | 3035
 100003 | 20161005 | 2016-10-05 10:51:29.906000+0000 | 6 |  142.8354 |  579
 100003 | 20161005 | 2016-10-05 10:51:32.918000+0000 | 2 |   9.87186 | 6030

(3 rows)
[donghua@localhost dc1]$ cqlsh 127.0.0.3 9042 -e "select * from mykeyspace.t1 limit 3"

 a      | b        | c                               | d | e         | f
--------+----------+---------------------------------+---+-----------+------
 100003 | 20161005 | 2016-10-05 10:51:29.157000+0000 | 2 | 115.40718 | 3035
 100003 | 20161005 | 2016-10-05 10:51:29.906000+0000 | 6 |  142.8354 |  579
 100003 | 20161005 | 2016-10-05 10:51:32.918000+0000 | 2 |   9.87186 | 6030

(3 rows)