Building a Local Test Environment

The following instructions will guide you through installing a Riak CS test environment. This guide does not cover system/service tuning and it does not attempt to optimize your installation for your particular architecture.

If you want to build a testing environment with a minimum of configuration, there is an option for Building a Virtual Testing Environment.

Installing Your First Node

You should complete the following preparatory steps before installing and running Riak and Riak CS.

Step 1: Raise your system’s open file limits

Riak can consume a large number of open file handles during normal operation. See the Open Files Limit document for more information on how to increase your system’s open files limit.

If you are the root user, you can increase the system’s open files limit for the current session with this command:

ulimit -n 65536

For this setting to persist in most Linux distributions, you also need to save it for the root and riak users in /etc/security/limits.conf:

# ulimit settings for Riak CS
root soft nofile 65536
root hard nofile 65536
riak soft nofile 65536
riak hard nofile 65536

For Mac OS X, consult the open files limit documentation.

Step 2: Download and install packages

This guide uses curl for downloading packages and interacting with the Riak CS API, so let’s make sure that it’s installed:

sudo apt-get install -y curl

Note: If you’re running Riak CS on a non-Debian/Ubuntu OS, substitute the appropriate CLI commands.

If you are running Ubuntu 11.10 or later, you will also need the libssl0.9.8 package. See Installing on Debian and Ubuntu for more information.

sudo apt-get install -y libssl0.9.8

Now, grab the appropriate packages: Riak, Riak CS, and Stanchion. See Download Riak and Download Riak CS. You can skip Riak CS Control for now.

Once you have the packages, install them per the instructions below.

First, install Riak

The following links provide platform-specific instructions for installing Riak.

Do not attempt to configure or start Riak until step 3 in this document.

Next, install Riak CS

For Mac OS X:

curl -O http://s3.amazonaws.com/downloads.basho.com/<riak-cs-os-x.tar.gz>
tar -xvzf <riak-cs-os-x.tar.gz>

Replace <riak-cs-os-x.tar.gz> with the actual filename for the package you are installing.

For RedHat Enterprise distributions (and similar):

rpm -Uvh <riak-cs-package.rpm>

Replace <riak-cs-package.rpm> with the actual filename for the package you are installing.

Ubuntu distributions and similar:

sudo dpkg -i <riak-cs-package.deb>

Replace <riak-cs-package.deb> with the actual filename for the package you are installing.

Finally, install Stanchion

For Mac OS X:

curl -O http://s3.amazonaws.com/downloads.basho.com/<stanchion-os-x.tar.gz>
tar -xvzf <stanchion-os-x.tar.gz>

Replace <stanchion-os-x.tar.gz> with the actual filename for the package you are installing.

For RedHat Enterprise distributions (and similar):

sudo rpm -Uvh <stanchion-package.rpm>

Replace <stanchion-package.rpm> with the actual filename for the package you are installing.

For Ubuntu distributions:

sudo dpkg -i <stanchion-package.deb>

Replace <stanchion-package.deb> with the actual filename for the package you are installing.

Step 3: Set service configurations and start the services

You will need to make changes to several configuration files.

/etc/riak/riak.conf

Be sure the storage backend is not set:

## Delete this line
storage_backend = . . .

And that the default bucket properties allow siblings:

## Append this line at the end of the file
buckets.default.allow_mult = true

Next, you need to expose the necessary Riak CS modules to Riak and instruct Riak to use the custom backend provided by Riak CS. You’ll have to use the old-style /etc/riak/advanced.config for these settings. The file should look like:

[
 {riak_kv, [
              {add_paths, ["/usr/lib/riak-cs/lib/riak_cs-2.1.1/ebin"]},
              {storage_backend, riak_cs_kv_multi_backend},
              {multi_backend_prefix_list, [{<<"0b:">>, be_blocks}]},
              {multi_backend_default, be_default},
              {multi_backend, [
                  {be_default, riak_kv_eleveldb_backend, [
                      {total_leveldb_mem_percent, 30},
                      {data_root, "/var/lib/riak/leveldb"}
                  ]},
                  {be_blocks, riak_kv_bitcask_backend, [
                      {data_root, "/var/lib/riak/bitcask"}
                  ]}
              ]}
  ]}
].
Note on OS-specific paths

The path for add_paths may be /usr/lib/riak-cs or /usr/lib64/riak-cs depending on your operating system.

Next, set your interface IP addresses in the riak.conf file. In a production environment, you’d likely have multiple NICs, but for this test cluster, assume one NIC with an example IP address of 10.0.2.10.

Change the following lines in /etc/riak/riak.conf

listener.http.internal = 127.0.0.1:8098
listener.protobuf.internal = 127.0.0.1:8087

to

listener.http.internal = 10.0.2.10:8098
listener.protobuf.internal = 10.0.2.10:8087

/etc/riak-cs/riak-cs.conf

Change the following lines in /etc/riak-cs/riak-cs.conf

listener = 127.0.0.1:8080
riak_host = 127.0.0.1:8087
stanchion_host = 127.0.0.1:8085

to

listener = 10.0.2.10:8080
riak_host = 10.0.2.10:8087
stanchion_host = 10.0.2.10:8085

The listener could also be set to 0.0.0.0if you prefer Riak CS to listen on all interfaces.

/etc/stanchion/stanchion.conf

Change the following lines in /etc/stanchion/stanchion.conf

listener = 127.0.0.1:8085
riak_host = 127.0.0.1:8087

to

listener = 10.0.2.10:8085
riak_host = 10.0.2.10:8087

Service names

Next, set your service names, using either use the local IP address for this or set hostnames. If you choose to set hostnames, you should ensure that the hostnames are resolvable by DNS or set in /etc/hosts on all nodes. Note: Service names require at least one period in the name.

Change the following line in /etc/riak/riak.conf

nodename = riak@127.0.0.1

to

nodename = riak@10.0.2.10

Then change the following line in /etc/riak-cs/riak-cs.conf

nodename = riak-cs@127.0.0.1

to

nodename = riak-cs@10.0.2.10

Change the following line in /etc/stanchion/stanchion.conf

nodename = stanchion@127.0.0.1

to

nodename = stanchion@10.0.2.10

Start the services

That is the minimum amount of service configuration required to start a complete node. To start the services, run the following commands in the appropriate /bin directories:

sudo riak start
sudo stanchion start
sudo riak-cs start

The order in which you start the services is important, as each is a dependency for the next. Make sure that you successfully start Riak before Stanchion and Stanchion before Riak CS.

You can check the liveness of your Riak CS installation and its connection to the supporting Riak node. If the Riak CS node is running, the following command should return PONG.

riak-cs ping

To check that the Riak CS node is communicating with its supporting Riak node, run a GET request against the riak-cs/ping endpoint of the Riak CS node. For example:

curl http://localhost:8080/riak-cs/ping

Step 4: Create the admin user

Creating the admin user is an optional step, but it’s a good test of our new services. Creating a Riak CS user requires two inputs:

  1. Name — A URL-encoded string, e.g. admin%20user
  2. Email — A unique email address, e.g. admin@admin.com

To create an admin user, we need to grant permission to create new users to the anonymous user. This configuration setting is only required on a single Riak CS node.

Add this entry to /etc/riak-cs/riak-cs.conf:

anonymous_user_creation = on

Then run sudo riak-cs stop && sudo riak-cs start to put the new config setting into effect.

We can create the admin user with the following curl command, on the same Riak CS machine where the anonymous_user_creation configuration option was enabled:

curl -XPOST http://localhost:8080/riak-cs/user \
  -H 'Content-Type: application/json' \
  -d '{"email":"admin@admin.com", "name":"admin"}'

The output of this command will be a JSON object that looks something like this:

{
  "email": "admin@admin.com",
  "display_name": "admin",
  "name": "admin user",
  "key_id": "5N2STDSXNV-US8BWF1TH",
  "key_secret": "RF7WD0b3RjfMK2cTaPfLkpZGbPDaeALDtqHeMw==",
  "id": "4b823566a2db0b7f50f59ad5e43119054fecf3ea47a5052d3c575ac8f990eda7"
}

The user’s access key and secret key are returned in the key_id and key_secret fields respectively. Take note of these keys as they will be required in the testing step.

In this case, those keys are:

  • Access key — 5N2STDSXNV-US8BWF1TH
  • Secret key – RF7WD0b3RjfMK2cTaPfLkpZGbPDaeALDtqHeMw==

You can use this same process to create additional Riak CS users. To make this user the admin user, we set these keys in the Riak CS riak-cs.conf and stanchion.conf files.

Note on admin keys

The same admin keys will need to be set on all nodes of the cluster.

Change the following lines in /etc/riak-cs/riak-cs.conf on all Riak CS machines:

admin.key = admin-key
admin.secret = admin-secret

to

admin.key = 5N2STDSXNV-US8BWF1TH
admin.secret = RF7WD0b3RjfMK2cTaPfLkpZGbPDaeALDtqHeMw==

Note: Make sure to set the anonymous_user_creation setting to off at this point.

Change the following lines in /etc/stanchion/stanchion.conf

admin.key = admin-key
admin.secret = admin-secret

to

admin.key = 5N2STDSXNV-US8BWF1TH
admin.secret = RF7WD0b3RjfMK2cTaPfLkpZGbPDaeALDtqHeMw==

Now we have to restart the services for the change to take effect:

sudo stanchion stop && sudo stanchion start
sudo riak-cs stop && sudo riak-cs start

Installing Additional Nodes

The process for installing additional nodes is identical to installing your first node with two exceptions:

  1. Stanchion only needs to be installed on your first node; there is no need to install it again on each node. The stanchion_ip setting in your Riak CS app.config files should be set to the stanchion_ip from your first node.

  2. To add additional nodes to the Riak cluster, use the following command

    sudo riak-admin cluster join riak@10.0.2.10

    where riak@10.0.2.10 is the Riak node name set in your first node’s /etc/riak/vm.args file

You will then need to verify the cluster plan with the riak-admin cluster plan command, and commit the cluster changes with riak-admin cluster commit to complete the join process. More information is available in the Command Line Tools documentation.

Note

Riak CS is not designed to function directly on TCP port 80, and should not be operated in a manner which exposes it directly to the public internet. Instead, consider a load balancing solution, such as a dedicated device, HAProxy, or Nginx between Riak CS and the outside world.

Once you have completed this step, You can progress to testing the Riak CS installation using s3cmd.