security added for mongodb

mongodb/README.md

@@ -5,22 +5,32 @@ In this example we demonstrate how we can orchestrate the deployment of a produc
 
 1) Deploying a N node MongoDB cluster, which has N shards and N replication nodes.
 2) Scale out capability. Expand the Cluster by adding nodes to the cluster.
+3) Security, All the mongodb process are secured using the best practices.
 
 ###Deployment Architecture.
 
 To better explain the deployment architecture let's take an example where we are deploying a 3 node MongoDB cluster ( Minimum recommended by MongoDB).
+
 The way Ansible configures the three nodes is as follows:
 
 1) Install the mongodb software on all nodes.
+
 2) Creates 3 replication sets, with one primary on each node and the rest two acting as secondaries.
-3) Configures MongodDB configuration servers as listed in the inventory section[mongocservers]. Recommended number is 3, so it can be the same three servers as the datanodes.
+
+3) Configures MongodDB configuration DB servers as listed in the inventory section[mongocservers]. Recommended number is 3, so it can be the same three servers as the datanodes.
+
 4) Configures a Mongos server as listed in the inventory file [mongosservers].
-5) Adds 3 Shards each belonging to individual relocation sets.
+
+5) Adds 3 Shards each belonging to individual replication sets.
+
+6) All the processes, mongod,mogos are secured using the keyfiles.
 
 Once the cluster is deployed, if we want to scale the cluster, Ansible configures it as follows:
 
 1) Install the MongoDB application on the new node.
+
 2) Configure the replication set with primary as the new node and the secondaries as listed in the inventory file [replicationservers]. ( don't forget to add the new node also in the replicationservers section]
+
 3) Adds a new shard to the mongos service pointing to the new replication set.
 
 ###The following example deploys a three node MongoDB Cluster
@@ -31,19 +41,19 @@ The inventory file looks as follows:
 		[mongoservers]
 		mongo1
 		mongo2
-                  mongo3
+                mongo3
 
 		#The list of servers where replication should happen, by default include all servers
 		[replicationservers]
 		mongo1
 		mongo2
-                  mongo3
+                mongo3
 
 		#The list of mongodb configuration servers, make sure it is 1 or 3
 		[mongocservers]
 		mongo1
 		mongo2
-                  mongo3
+                mongo3
 
 		#The list of servers where mongos servers would run. 
 		[mongosservers]
@@ -53,7 +63,7 @@ Build the site with the following command:
 
 		ansible-playbook -i hosts site.yml
 
-##Verification
+###Verifying the deployed MongoDB Cluster
 
 Once completed we can check replication set availibitly by connecting to individual primary replication set nodes, 'mongo --host <ip host> --port <port number> 
 and issue the command to query the status of replication set, we should get a similar output.
@@ -105,6 +115,70 @@ and issue the following command to get the status of the Shards.
   		databases:
 			{  "_id" : "admin",  "partitioned" : false,  "primary" : "config" }
 
+We can also make sure the Sharding works by creating a database and collection and populate it with documents and check if the chunks of the collection are balanced equally across nodes.
+
+The above mentioned steps can be tested as follows:
+
+1) Once the Sharded cluster is ready, create a new database and an admin user for the database. This is done from the mongos machine.
+
+		/usr/bin/mongo localhost:8888/admin -u admin -p 123456
+		mongos> use test
+		switched to db test
+		
+		mongos> db.addUser('admin','123456')
+		{
+			"user" : "admin",
+			"readOnly" : false,
+			"pwd" : "95ec4261124ba5951720b199908d892b",
+			"_id" : ObjectId("51519f349cd3a93ca7e17909")
+		}
+2) Once the DB and the user is created, create a collection and poplulate documents, This deployment add a script to the /tmp location of the mongos server which adds a new collection and 100,000 documents.
+
+		$/usr/bin/mongo localhost:8888/test -u admin -p 123456 /tmp/testsharding.js 
+
+3) After the document's are populated, we have to enable sharding on the database and the collection. which can be done as follows:
+		
+		$/usr/bin/mongo localhost:8888/admin -u admin -p 123456
+		mongos> db.runCommand( { enableSharding : "test" } )
+			{ "ok" : 1 }
+		mongos> db.runCommand( { shardCollection : "test.test_collection", key : {"number":1} })
+			{ "collectionsharded" : "test.test_collection", "ok" : 1 }
+		mongos> sh.status()
+			--- Sharding Status --- 
+  			sharding version: { "_id" : 1, "version" : 3 }
+  			shards:
+			{  "_id" : "bensible",  "host" : "bensible/bensible:20103,web2:20103,web3:20103" }
+			{  "_id" : "web2",  "host" : "web2/bensible:20102,web2:20102,web3:20102" }
+			{  "_id" : "web3",  "host" : "web3/bensible:20101,web2:20101,web3:20101" }
+  			databases:
+			{  "_id" : "admin",  "partitioned" : false,  "primary" : "config" }
+			{  "_id" : "test",  "partitioned" : true,  "primary" : "bensible" }
+			
+				test.test_collection chunks:
+
+					web2	1
+					bensible	19
+
+
+4) In the above example we can see the chunks being balanced across nodes. After a few minutes if we excute the same command 'sh.status()'
+we will see the below output, which shows all the chunks being balanced across the three nodes.
+
+		mongos> sh.status()
+			--- Sharding Status --- 
+  			sharding version: { "_id" : 1, "version" : 3 }
+  			shards:
+			{  "_id" : "bensible",  "host" : "bensible/bensible:20103,web2:20103,web3:20103" }
+			{  "_id" : "web2",  "host" : "web2/bensible:20105,web2:20105,web3:20105" }
+			{  "_id" : "web3",  "host" : "web3/bensible:20102,web2:20102,web3:20102" }
+  			databases:
+			{  "_id" : "admin",  "partitioned" : false,  "primary" : "config" }
+			{  "_id" : "test",  "partitioned" : true,  "primary" : "web3" }
+			
+				test.test_collection chunks:
+				
+				bensible	7
+				web2	6
+				web3	7
 			
 			
 
@@ -117,20 +191,20 @@ To add a new node to the configured MongoDb Cluster, setup the inventory file as
 		[mongoservers]
 		mongo1
 		mongo2
-                  mongo3
+                mongo3
 		mongo4
 
 		#The list of servers where replication should happen, by default include all servers
 		[replicationservers]
 		mongo4
 		mongo1
-                  mongo2
+                mongo2
 
 		#The list of mongodb configuration servers, make sure it is 1 or 3
 		[mongocservers]
 		mongo1
 		mongo2
-                  mongo3
+                mongo3
 
 		#The list of servers where mongos servers would run. 
 		[mongosservers]
@@ -140,5 +214,28 @@ Make sure you have the new node added in the replicationservers section and exec
 
 		ansible-playbook -i hosts playbooks/addnode.yml -e servername=mongo4
 
-Verification can be done using the same steps mentioned above.
+###Verification.
+
+The verification of the newly added node can be as easy checking the sharding status and see the chunks being rebalanced to the newly added node.
+
+			$/usr/bin/mongo localhost:8888/admin -u admin -p 123456
+			mongos> sh.status()
+				--- Sharding Status --- 
+  				sharding version: { "_id" : 1, "version" : 3 }
+  			shards:
+			{  "_id" : "bensible",  "host" : "bensible/bensible:20103,web2:20103,web3:20103" }
+			{  "_id" : "web2",  "host" : "web2/bensible:20105,web2:20105,web3:20105" }
+			{  "_id" : "web3",  "host" : "web3/bensible:20102,web2:20102,web3:20102" }
+			{  "_id" : "web4",  "host" : "web4/bensible:20101,web3:20101,web4:20101" }
+  			databases:
+			{  "_id" : "admin",  "partitioned" : false,  "primary" : "config" }
+			{  "_id" : "test",  "partitioned" : true,  "primary" : "bensible" }
+		
+			test.test_collection chunks:
+			
+				web4	3
+				web3	6
+				web2	6
+				bensible	5
+
     

mongodb/group_vars/all

@@ -10,10 +10,13 @@ mongos_port: 8888
 mongoc_port: 7777
 
 #The port prefix for mongod servers, the latter part is appended the playbook (the last octect of the ipaddress)
-mongodb_port_prefix: 201
+mongodb_port_prefix: 20
 
 #The directory prefix where the database files would be stored
 mongodb_datadir_prefix: /data/
 
 #The interface where the mongodb process should listen on.
-iface: eth0
+iface: eth1
+
+#The password for admin user
+mongo_admin_pass: 123456

mongodb/hosts

@@ -1,17 +1,23 @@
 #The site wide list of mongodb servers
 [mongoservers]
-web2
+mongo1
-web3
+mongo2
+mongo3
 
 #The list of servers where replication should happen, by default include all servers
 [replicationservers]
-web2
+mongo1
-web3
+mongo2
+mongo3
+
 
 #The list of mongodb configuration servers, make sure it is 1 or 3
 [mongocservers]
-web3
+mongo1
+mongo2
+mongo3
+
 
 #The list of servers where mongos servers would run. 
 [mongosservers]
-web3
+mongo1

mongodb/playbooks/addnode.yml

@@ -1,11 +1,11 @@
 ---
 #This playbook is used to add a new node the mongodb cluster
 
-- hosts: mongoservers
+- hosts: all 
-- hosts: replicationservers
-- hosts: mongosservers
-- hosts: ${servername}
   tasks:
    - include: ../roles/common/tasks/main.yml
+
+- hosts: ${servername}
+  tasks: 
    - include: ../roles/mongod/tasks/main.yml
    - include: ../roles/mongod/tasks/addshard.yml

mongodb/roles/common/tasks/main.yml

@@ -12,3 +12,4 @@
   with_items:
    - mongo-10gen
    - mongo-10gen-server
+   - bc

mongodb/roles/mongoc/files/secret

@@ -0,0 +1,3 @@
+qGO6OYb64Uth9p9Tm8s9kqarydmAg1AUdgVz+ecjinaLZ1SlWxXMY1ug8AO7C/Vu
+D8kA3+rE37Gv1GuZyPYi87NSfDhKXo4nJWxI00BxTBppmv2PTzbi7xLCx1+8A1uQ
+4XU0HA

mongodb/roles/mongoc/tasks/main.yml

@@ -16,6 +16,17 @@
 - name: Create the mongo configuration server file
   template: src=../roles/mongoc/templates/mongoc.conf.j2 dest=/etc/mongoc.conf
 
+- name: Create the script to add the admin user 
+  template: src=../roles/mongoc/templates/adduser.j2 dest=/tmp/adduser.js
+
+- name: Copy the keyfile for authentication
+  copy: src=../roles/mongod/files/secret dest=${mongodb_datadir_prefix}/secret owner=mongod group=mongod mode=0400
+
 - name: Start the mongo configuration server service
   command: creates=/var/lock/subsys/mongoc /etc/init.d/mongoc start
 
+- name: pause
+  pause: seconds=20
+
+- name: add the admin user
+  shell: /usr/bin/mongo localhost:7777/admin /tmp/adduser.js

mongodb/roles/mongoc/templates/adduser.j2

@@ -0,0 +1 @@
+db.addUser('admin','{{ mongo_admin_pass }}')

mongodb/roles/mongoc/templates/mongoc.conf.j2

@@ -10,7 +10,7 @@ fork = true
 port = {{ mongoc_port }}
 
 dbpath={{ mongodb_datadir_prefix }}configdb
-
+keyFile={{ mongodb_datadir_prefix }}secret
 # location of pidfile
 pidfilepath = /var/run/mongoc.pid
 

mongodb/roles/mongod/files/secret

@@ -0,0 +1,3 @@
+qGO6OYb64Uth9p9Tm8s9kqarydmAg1AUdgVz+ecjinaLZ1SlWxXMY1ug8AO7C/Vu
+D8kA3+rE37Gv1GuZyPYi87NSfDhKXo4nJWxI00BxTBppmv2PTzbi7xLCx1+8A1uQ
+4XU0HA

mongodb/roles/mongod/tasks/addshard.yml

@@ -12,7 +12,7 @@
   with_items: ${groups.mongosservers}
 
 - name: Add the shard to the mongos 
-  shell: /usr/bin/mongo --port ${mongos_port} /tmp/shard_init_${inventory_hostname}.js
+  shell: /usr/bin/mongo localhost:8888/admin -u admin -p ${mongo_admin_pass} /tmp/shard_init_${inventory_hostname}.js
   delegate_to: $item
   with_items: ${groups.mongosservers}
 

mongodb/roles/mongod/tasks/main.yml

@@ -21,7 +21,7 @@
   with_items: ${groups.replicationservers}
 
 - name: Add the iptable rule to allow traffice dynamically
-  shell: iptables -I INPUT 2  -p tcp --dport ${mongodb_port_prefix}${result.stdout} -j ACCEPT
+  shell: sleep `echo $RANDOM/1000 | bc`; iptables -I INPUT 1  -p tcp --dport ${mongodb_port_prefix}${result.stdout} -j ACCEPT
   delegate_to: $item
   with_items: ${groups.replicationservers}
 
@@ -30,6 +30,10 @@
   delegate_to: $item
   with_items: ${groups.replicationservers}
 
+- name: Copy the keyfile for authentication
+  copy: src=../roles/mongod/files/secret dest=${mongodb_datadir_prefix}/secret owner=mongod group=mongod mode=0400
+
+
 - name: Start the mongodb service
   command: creates=/var/lock/subsys/mongod-${inventory_hostname} /etc/init.d/mongod-${inventory_hostname} start
   delegate_to: $item
@@ -38,6 +42,9 @@
 - name: Create the file to initialize the mongod replica set
   template: src=../roles/mongod/templates/repset_init.j2 dest=/tmp/repset_init.js
 
+- name: Pause for a while
+  pause: seconds=20
+
 - name: Initialize the replication set
   shell: /usr/bin/mongo --port "$mongodb_port_prefix${result.stdout}" /tmp/repset_init.js
 

mongodb/roles/mongod/templates/mongod.conf.j2

@@ -12,6 +12,7 @@ fork = true
 port = {{ mongodb_port_prefix }}{{ result.stdout }}
 
 dbpath={{ mongodb_datadir_prefix }}mongo-{{ inventory_hostname }}
+keyFile={{ mongodb_datadir_prefix }}/secret
 
 # location of pidfile
 pidfilepath = /var/run/mongod.pid

mongodb/roles/mongos/files/secret

@@ -0,0 +1,3 @@
+qGO6OYb64Uth9p9Tm8s9kqarydmAg1AUdgVz+ecjinaLZ1SlWxXMY1ug8AO7C/Vu
+D8kA3+rE37Gv1GuZyPYi87NSfDhKXo4nJWxI00BxTBppmv2PTzbi7xLCx1+8A1uQ
+4XU0HA

mongodb/roles/mongos/tasks/main.yml

@@ -13,6 +13,13 @@
 - name: Create the mongos configuration file
   template: src=../roles/mongos/templates/mongos.conf.j2 dest=/etc/mongos.conf
 
+- name: Copy the keyfile for authentication
+  copy: src=../roles/mongod/files/secret dest=${mongodb_datadir_prefix}/secret owner=mongod group=mongod mode=0400
+
 - name: Start the mongos service
   command: creates=/var/lock/subsys/mongos /etc/init.d/mongos start
+- name: pause
+  pause: seconds=20
 
+- name: copy the file for shard test 
+  template: src=../roles/mongos/templates/testsharding.j2 dest=/tmp/testsharding.js

mongodb/roles/mongos/templates/mongos.conf.j2

@@ -19,4 +19,5 @@ port = {{ mongos_port }}
 
 # location of pidfile
 pidfilepath = /var/run/mongodb/mongos.pid
+keyFile={{ mongodb_datadir_prefix }}/secret
 chunkSize={{ mongos_chunk_size }}

mongodb/roles/mongos/templates/testsharding.j2

@@ -0,0 +1,12 @@
+people = ["Marc", "Bill", "George", "Eliot", "Matt", "Trey", "Tracy", "Greg", "Steve", "Kristina", "Katie", "Jeff"];
+
+for(var i=0; i<100000; i++){
+                             name = people[Math.floor(Math.random()*people.length)];
+                             user_id = i;
+                             boolean = [true, false][Math.floor(Math.random()*2)];
+                             added_at = new Date();
+                             number = Math.floor(Math.random()*10001);
+                             db.test_collection.save({"name":name, "user_id":user_id, "boolean": boolean, "added_at":added_at, "number":number });
+                            }
+db.test_collection.ensureIndex({number:1})
+