Replication and snitches in Cassandra

In the previous post we’ve seen how data distribution in Cassandra works. But what if one of the nodes fails? If you want to ensure fault tolerance, you need to enable data replication. When you create a keyspace in Cassandra, you must decide the replica placement strategy: the number of replicas and how those replicas are distributed across nodes in the cluster. The replication strategy relies on the cluster-configured snitch to help it determine the physical location of nodes and their proximity to each other.

The total number of replicas across the cluster is often referred to as the replication factor.  A replication factor of 1 means there is only one copy of a row in a cluster. A replication factor of 2 means there are two copies of a row stored in a cluster. This replication factor is controlled at a key space level.

The different types of replication strategies are:

Simple strategy: This strategy places the original row on a node determined by a practitioner and then a copy of the row is placed on the next node walking clock-wise in the cluster.

Network topology strategy:  This is more sophisticated strategy than the simple strategy. Network topology strategy gives you more control over where the rows are placed in your database cluster. This allows you to place different rows on racks in a data center or to carry out geographical replication between multiple data centers. You can specify the replication strategy when creating a keyspace like:

CREATE KEYSPACE your_keySpace WITH strategy_class = 'NetworkTopologyStrategy' AND strategy_options:DC1 = 4; where DC1 = 4 specifies the replication factor.

Cassandra uses a snitch to define how nodes are grouped together within the overall network topology. This again can be defined in the configuration file cassandra.yaml

The four basic snitches you can define are:

Simple snitch: This is the default snitch used and uses a simple strategy of placing the copy of the row on the next available node walking clockwise through the nodes.

Rack inferring snitch:  The rack inferring snitch automatically tries to place copies of rows on different racks in your data center. The rack inferring snitch uses the node's IP address to infer the network topology. The second unit of the IP address is used to identify the data center where a node is location and the third unit identifies the rack.

Property file snitch: Is used to determine the location of the nodes by referring to the cassandra-topology.properties file.

EC2 snitch: If you want to use cassandra in the cloud, you have to use the EC2 snitch which uses the AWS API to request the region and the respective availability zone.

Cassandra Architecture - Data Partitioning

Before starting a Cassandra cluster, you must choose how the data will be divided across the nodes in the cluster. This involves choosing a partitioner for the cluster. Cassandra uses a ring architecture. The ring is divided up into ranges equal to the number of nodes, with each node being responsible for one or more ranges of the overall data. Before a node can join the ring, it must be assigned a token. The token determines the node’s position on the ring and the range of data it is responsible for.

Once the partitioner is chosen it is unlikely to change the configuration choice without reloading all the data. This makes it very important to choose and configure the correct partitioner before initializing the cluster.

The important distinction between the partitioners is order preservation (OP). Users can define their own partitioners by implementing IPartitioner, or they can use one of the native partitioners.

Random Partitioner

RandomPartitioner  is the default choice for cassandra as it uses an MD5 hash function to map keys into tokens. These keys will evenly distribute across the clusters. The row key determines where the node placement.  Consistent hashing algorithm used by Random partioning ensures that when nodes are added to the cluster, the minimum possible set of data is affected. The hashing algorithm creates an MD5 hash value of the row key ranging from 0 to 2*127. Then nodes in the cluster are assigned a token that represents the hash value in the above mentioned range. This value determines the row keys to be placed in the node. For e.g the below given row with row key ‘Prajeesh’ is assigned a hash key like 98002736AD65AB which determines the node that holds the range to store the row.

Prajeesh

India

Scrum Master

Prowareness

Notice that the keys are not in order. With RandomPartitioner, the keys are evenly distributed across the ring using hashes, but you sacrifice order, which means any range query needs to query all nodes in the ring.

Ordered Preserving Partitioners

The Order Preserving Partitioners preserve the order of the row keys as they are mapped into the token space. This allows range scans over rows, meaning you can scan rows as though you were moving a cursor through a traditional index. For example, if your application has user names as the row key, you can scan rows for users whose names fall between Albert and Amy. This type of query would not be possible with randomly partitioned row keys, since the keys are stored in the order of their MD5 hash (not sequentially).

An advantage of using OPP is that the range queries are simplified since the query need not consult each node in the ring the fetch the data. It can directly visit the node based on the order of row keys.

A disadvantage of using OPP is that the ring becomes unstable over a time if your application tends to write or update a sequential block of rows at a time, then the writes will not be distributed across the cluster, putting it all to a node. This makes one node holding more data than the rest disturbing the even distribution of data across nodes. 

Using Apache Cassandra with .NET - Part 2

One major difference between Cassandra and a traditional relational database is that Cassandra supports a variable amount of columns per rows in any given column family. In this post we will insert a column to the column family, "Tweets" The tweet column family is of contains one row with the text “sample tweet text1”. We will also see the test case for a null value insert exception.

[TestMethod]

public void InsertOnSubmitInsertsANewValueToTheColumnFamily()

{

    var key = "1".ToCassandraByte();

    const string columnName = "text";

    const string value = "sample tweet text 1";

    using (var context = new CassandraContext("localhost", 9160, "Twitter"))

    {

        context.Column.DeleteOnSubmit(x => x.ColumnFamily == "Tweets" && x.Key == key);

        context.SubmitChanges();

        var column = new Column().SetNameValue(columnName, value);

        context.InsertOnSubmit("Tweets", key, column);

        context.SubmitChanges();

        var tweet = (from x in context.ColumnList

                        .Where(x => x.ColumnFamily == "Tweets" && x.Key == key)

                    select x.ToObject<Tweet>()).FirstOrDefault();

        Assert.AreEqual(value, tweet.text);

        context.Column.DeleteOnSubmit(x => x.ColumnFamily == "Tweets" && x.Key == key);

        context.SubmitChanges();

    }

}

[TestMethod]

[ExpectedException(typeof(Thrift.TApplicationException))]

public void InsertNullValueTest()

{

    var key = "1".ToCassandraByte();

    const string columnName = "text";

    using (var context = new CassandraContext("localhost", 9160, "Twitter"))

    {

        var column = new Column()

            .SetNameValue(columnName, null);

        context.InsertOnSubmit("Tweet", key, column);

        context.SubmitChanges();

    }

}

Configuring data consistency options in Cassandra

Cassandra shows excellent performance in data distribution and availability. In Cassandra, consistency refers to how up-to-date and synchronized a row of data is on all of its replicas. Cassandra extends the concept of eventual consistency by offering tunable consistency. For any given read or write operation, the client application decides how consistent the requested data should be. In addition to tunable consistency, Cassandra has a number of built-in repair mechanisms to ensure that data remains consistent across replicas.

Cassandra supports the following write consistency levels.

• ANY, having the lowest consistency with high availability. This mentions that a write must be written to at least one node. If all replica nodes for the given row key are down, the write can still succeed once a hinted handoff has been written.
• ALL, having the highest consistency with lowest availability. A write must be written to the commit log and memory table on all replica nodes in the cluster for that row key.
• QUORUM is a good middle-ground ensuring strong consistency, yet still tolerating some level of failure. A write must be written to the commit log and memory table on a quorum of replica nodes.
• ONE: A write must be written to the commit log and memory table of at least one replica node.
• LOCAL_QUORUM: A write must be written to the commit log and memory table on a quorum of replica nodes in the same data center as the coordinator node. Avoids latency of inter-data center communication.
• EACH_QUORUM: A write must be written to the commit log and memory table on a quorum of replica nodes in all data centers.

For read consistency Cassandra supports the below given levels

• ONE:  If latency is a top priority, consider a consistency level of ONE (only one replica node must successfully respond to the read or write request). There is a higher probability of stale data being read with this consistency level (as the replicas contacted for reads may not always have the most recent write).
• ANY: If it is an absolute requirement that a write never fail, you may also consider a write consistency level of ANY. This consistency level has the highest probability of a read not returning the latest written values (see hinted handoff).
• QUORUM: Returns the record with the most recent timestamp once a quorum of replicas has responded.

Creating a Cassandra connection configuration in .NET

[TestMethod]

public void CassandraConnectionConfigBuilderTest()

{

    var hosts = new[] { "a", "b", "c" };

    var builder = new CassandraConnectionConfigBuilder

    {

        Hosts = hosts,

        Port = 9160,

        ConsistencyLevel = ConsistencyLevel.QUORUM,

        Timeout = TimeSpan.FromSeconds(100),

        IsFramed = true,

    };

    var config = new CassandraConnectionConfig(builder);

    CollectionAssert.AreEqual(hosts, config.Hosts);

    Assert.AreEqual(9160, config.Port);

    Assert.AreEqual(ConsistencyLevel.QUORUM, config.ConsistencyLevel);

    Assert.AreEqual(TimeSpan.FromSeconds(100), config.Timeout);

    Assert.AreEqual(true, config.IsFramed);

           

}

Using Apache Cassandra with .NET - Part 1

With the uninterrupted growth of voluminous amount of unstructured and semi-structured computing data, storage of information, support and maintenance has been the biggest challenge. Data of this type would take too much time and cost too much money to load into a relational database for analysis. The demands of huge data and elastic scaling with desired performance has led to concept of big data. Although big data doesn't refer to any specific quantity, the term is often used when speaking about petabytes and exabytes of data.

Apache Cassandra is a standout among the NoSQL/post-relational database solutions on the market for many reasons. Some of the core features of Cassandra are:

• Highly scalable peer-to-peer architecture based on the best of Amazon Dynamo and Google BigTable. Cassandra is the considered as the NoSQL leader when it comes to comfortably scaling to terabytes or petabytes of data.
• Increased database for both read and write operations via nodes and cluster. Data is replicated to multiple nodes to protect from loss during node failure, and new machines can be added incrementally while online to increase the capacity and data protection of your Cassandra cluster.
•  Ensure data safety due to its innovative append-only commit log. Users no longer have to trade off durability to keep up with immense write streams.
• Transparent fault detection and recovery using gossip protocols to detect machine failure and recover when a machine is brought back into the cluster – all without your application noticing.
• Cassandra offers support for multiple data centres by easy configuration options for setting copies of your data you want in each data centre.
• Cassandra offers caching on each of its nodes enabling ease of development.
• Incremental and dynamic expansion as Cassandra ring allows you to add nodes easily without manual migration of data needed from one to another.
• Cassandra runs on commodity machines and requires no expensive or special hardware.

Cassandra’s data model

The Cassandra data model has 4 main concepts which are cluster, keyspace, column family and super column.

• Cluster is the outermost structure in Cassandra (also called as ring). Cassandra database is specially designed to be spread across several machines functioning together that act as a single occurrence to the end user. Cassandra allocates data to nodes in the cluster by arranging them in a ring. Clusters contain many nodes (machines) and can contain multiple keyspaces.
• A keyspace is the top level element of a schema.  A keysapce is a Container for column families to group multiple column families, typically one per application.
• A column contains a name, value and timestamp. A column must have a name, and the name can be a static label (such as “name” or “email”) or it can be dynamically set when the column is created by your application.
• A column family contains multiple columns referenced by a row keys. Column families can (and should) define metadata about the columns, but the actual columns that make up a row are determined by the client application. Each row can have a different set of columns.

Configuring Cassandra using DataStax OpsCenter

DataStax OpsCenter is a visual management and monitoring solution for big data platforms designed to manage and monitor Apache Cassandra™ database clusters. Through an intuitive point-and-click interface, a user can understand the state of one or more clusters through a centralized dashboard that provides an at-a-glance view of the status, activity, and issues across all monitored clusters. Users can easily create new database objects, manage maintenance activities, and perform other database tasks such as backups in a visual manner.  The opscenter interface looks like.

You can add a keyspace to the available cluster as given below.

After adding the keyspace you can add a column family as.

Later you can also use the CIL to change the schema structure or add new data to the column family as given below.

Accessing Cassandra in .NET

Now let’s write some code to access the posts we stored in the posts column family. We’ll be using Cassandraemon,  a LINQ client for Cassandra Apache.

[TestMethod]

public void OpenAConnectionTest()

{

    using (var context = new CassandraContext("localhost", 9160, "Twitter"))

    {

        var products = from x in context.ColumnList

                        where x.ColumnFamily == "Tweets"

                        && x.Column == "text"

                        select x.ToObject<Tweet>();  

        Assert.IsTrue(products.Count() > 0);

    }

}