MongoDB - Configure KMIP backed mongod (with key roration)

[donghua@infrastructure shared]$ python pykmip_server.py

2016-11-17 10:00:11,753 - __main__ - INFO - Starting KMIP server

[donghua@database shared]$ mkdir -p /home/donghua/LAB5

# Configure encryption using a New Key https://docs.mongodb.com/manual/tutorial/configure-encryption/#key-manager

mongod --dbpath /home/donghua/LAB5 --logpath /home/donghua/LAB5/mongo.log --port 31260 --fork --enableEncryption --kmipServerName infrastructure.dbaglobe.com --kmipServerCAFile /home/donghua/shared/certs/ca.pem --kmipClientCertificateFile /home/donghua/shared/certs/client.pem

2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten] MongoDB starting : pid=5896 port=31260 dbpath=/home/donghua/LAB5 64-bit host=database.dbaglobe.com

2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten] db version v3.2.10

2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten] git version: 79d9b3ab5ce20f51c272b4411202710a082d0317

2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016

2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten] allocator: tcmalloc

2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten] modules: enterprise

2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten] build environment:

2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten]     distmod: ubuntu1604

2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten]     distarch: x86_64

2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten]     target_arch: x86_64

2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten] options: { net: { port: 31260 }, processManagement: { fork: true }, security: { enableEncryption: true, kmip: { clientCertificateFile: "/home/donghua/shared/certs/client.pem", serverCAFile: "/home/donghua/shared/certs/ca.pem", serverName: "infrastructure.dbaglobe.com" } }, storage: { dbPath: "/home/donghua/LAB5" }, systemLog: { destination: "file", path: "/home/donghua/LAB5/mongo.log" } }

2016-11-17T23:00:14.804+0800 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),extensions=[local=(entry=mongo_addWiredTigerEncryptors)],encryption=(name=AES256-CBC,keyid=".system"),

2016-11-17T23:00:14.819+0800 I STORAGE  [initandlisten] Created KMIP key with id: 1

2016-11-17T23:00:14.877+0800 I STORAGE  [initandlisten] Encryption key manager initialized using KMIP key with id: 1.

2016-11-17T23:00:14.878+0800 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/home/donghua/LAB5/diagnostic.data'

2016-11-17T23:00:14.879+0800 I NETWORK  [HostnameCanonicalizationWorker] Starting hostname canonicalization worker

2016-11-17T23:00:14.916+0800 I NETWORK  [initandlisten] waiting for connections on port 31260

# (Optional) Rotate the key https://docs.mongodb.com/manual/tutorial/rotate-encryption-key/

mongod --dbpath /home/donghua/LAB5 --logpath /home/donghua/LAB5/mongo.log --port 31260 --fork --enableEncryption --kmipRotateMasterKey --kmipServerName infrastructure.dbaglobe.com --kmipServerCAFile /home/donghua/shared/certs/ca.pem --kmipClientCertificateFile /home/donghua/shared/certs/client.pem

donghua@database:~/LAB5$ cat /home/donghua/LAB5/mongo.log

2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten] MongoDB starting : pid=5962 port=31260 dbpath=/home/donghua/LAB5 64-bit host=database.dbaglobe.com

2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten] db version v3.2.10

2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten] git version: 79d9b3ab5ce20f51c272b4411202710a082d0317

2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016

2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten] allocator: tcmalloc

2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten] modules: enterprise

2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten] build environment:

2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten]     distmod: ubuntu1604

2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten]     distarch: x86_64

2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten]     target_arch: x86_64

2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten] options: { net: { port: 31260 }, processManagement: { fork: true }, security: { enableEncryption: true, kmip: { clientCertificateFile: "/home/donghua/shared/certs/client.pem", rotateMasterKey: true, serverCAFile: "/home/donghua/shared/certs/ca.pem", serverName: "infrastructure.dbaglobe.com" } }, storage: { dbPath: "/home/donghua/LAB5" }, systemLog: { destination: "file", path: "/home/donghua/LAB5/mongo.log" } }

2016-11-17T23:07:05.250+0800 I -        [initandlisten] Detected data files in /home/donghua/LAB5 created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.

2016-11-17T23:07:05.250+0800 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),extensions=[local=(entry=mongo_addWiredTigerEncryptors)],encryption=(name=AES256-CBC,keyid=".system"),

2016-11-17T23:07:05.598+0800 I STORAGE  [initandlisten] Created KMIP key with id: 2

2016-11-17T23:07:05.685+0800 I STORAGE  [initandlisten] Rotated master encryption key from id 1 to id 2.

2016-11-17T23:07:05.685+0800 I CONTROL  [initandlisten] now exiting

2016-11-17T23:07:05.685+0800 I NETWORK  [initandlisten] shutdown: going to close listening sockets...

2016-11-17T23:07:05.685+0800 I NETWORK  [initandlisten] removing socket file: /tmp/mongodb-31260.sock

2016-11-17T23:07:05.685+0800 I NETWORK  [initandlisten] shutdown: going to flush diaglog...

2016-11-17T23:07:05.685+0800 I NETWORK  [initandlisten] shutdown: going to close sockets...

2016-11-17T23:07:05.685+0800 I STORAGE  [initandlisten] WiredTigerKVEngine shutting down

2016-11-17T23:07:05.698+0800 I STORAGE  [initandlisten] shutdown: removing fs lock...

2016-11-17T23:07:05.698+0800 I CONTROL  [initandlisten] dbexit:  rc: 0

# Start MongoDB again

mongod --dbpath /home/donghua/LAB5 --logpath /home/donghua/LAB5/mongo.log --port 31260 --fork --enableEncryption --kmipServerName infrastructure.dbaglobe.com --kmipServerCAFile /home/donghua/shared/certs/ca.pem --kmipClientCertificateFile /home/donghua/shared/certs/client.pem

donghua@database:~/LAB5$ cat /home/donghua/LAB5/mongo.log

2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten] MongoDB starting : pid=6004 port=31260 dbpath=/home/donghua/LAB5 64-bit host=database.dbaglobe.com

2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten] db version v3.2.10

2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten] git version: 79d9b3ab5ce20f51c272b4411202710a082d0317

2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016

2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten] allocator: tcmalloc

2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten] modules: enterprise

2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten] build environment:

2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten]     distmod: ubuntu1604

2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten]     distarch: x86_64

2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten]     target_arch: x86_64

2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten] options: { net: { port: 31260 }, processManagement: { fork: true }, security: { enableEncryption: true, kmip: { clientCertificateFile: "/home/donghua/shared/certs/client.pem", serverCAFile: "/home/donghua/shared/certs/ca.pem", serverName: "infrastructure.dbaglobe.com" } }, storage: { dbPath: "/home/donghua/LAB5" }, systemLog: { destination: "file", path: "/home/donghua/LAB5/mongo.log" } }

2016-11-17T23:07:55.962+0800 I -        [initandlisten] Detected data files in /home/donghua/LAB5 created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.

2016-11-17T23:07:55.962+0800 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),extensions=[local=(entry=mongo_addWiredTigerEncryptors)],encryption=(name=AES256-CBC,keyid=".system"),

2016-11-17T23:07:56.151+0800 I STORAGE  [initandlisten] Encryption key manager initialized using KMIP key with id: 2.

2016-11-17T23:07:56.151+0800 I CONTROL  [initandlisten]

2016-11-17T23:07:56.151+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.

2016-11-17T23:07:56.151+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'

2016-11-17T23:07:56.151+0800 I CONTROL  [initandlisten]

2016-11-17T23:07:56.151+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.

2016-11-17T23:07:56.151+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'

2016-11-17T23:07:56.151+0800 I CONTROL  [initandlisten]

2016-11-17T23:07:56.153+0800 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/home/donghua/LAB5/diagnostic.data'

2016-11-17T23:07:56.153+0800 I NETWORK  [initandlisten] waiting for connections on port 31260

2016-11-17T23:07:56.153+0800 I NETWORK  [HostnameCanonicalizationWorker] Starting hostname canonicalization worker

 

 

MongoDB Enterprise > db.getSisterDB('admin').runCommand({getCmdLineOpts: 1})

{

        "argv" : [

                "mongod",

                "--dbpath",

                "/home/donghua/LAB5",

                "--logpath",

                "/home/donghua/LAB5/mongo.log",

                "--port",

                "31260",

                "--fork",

                "--enableEncryption",

                "--kmipServerName",

                "infrastructure.dbaglobe.com",

                "--kmipServerCAFile",

                "/home/donghua/shared/certs/ca.pem",

                "--kmipClientCertificateFile",

                "/home/donghua/shared/certs/client.pem"

        ],

        "parsed" : {

                "net" : {

                        "port" : 31260

                },

                "processManagement" : {

                        "fork" : true

                },

                "security" : {

                        "enableEncryption" : true,

                        "kmip" : {

                                "clientCertificateFile" : "/home/donghua/shared/certs/client.pem",

                                "serverCAFile" : "/home/donghua/shared/certs/ca.pem",

                                "serverName" : "infrastructure.dbaglobe.com"

                        }

                },

                "storage" : {

                        "dbPath" : "/home/donghua/LAB5"

                },

                "systemLog" : {

                        "destination" : "file",

                        "path" : "/home/donghua/LAB5/mongo.log"

                }

        },

        "ok" : 1

}

=====================================================

[donghua@infrastructure shared]$ sudo pip install PyKMIP==0.4.0

[donghua@infrastructure shared]$ cat pykmip_server.py

#!/usr/bin/python

# this file is a thin wrapper around the PyKMIP server

# which is required for some encrypted storage engine tests

import logging

from kmip.services.kmip_server import KMIPServer

def main():

    logger = logging.getLogger(__name__)

    server = KMIPServer(

        host="infrastructure.dbaglobe.com",

        port=5696,

        keyfile="/home/donghua/shared/certs/server.pem",

        certfile="/home/donghua/shared/certs/server.pem",

        cert_reqs="CERT_REQUIRED",

        ssl_version="PROTOCOL_TLSv1",

        ca_certs="/home/donghua/shared/certs/ca.pem",

        do_handshake_on_connect=True,

        suppress_ragged_eofs=True)

    logger.info("Starting KMIP server")

    try:

        server.serve()

    except Exception as e:

        logger.info('Exception received while serving: {0}'.format(e))

    finally:

        server.close()

    logger.info("Stopping KMIP server")

if __name__ == '__main__':

    main()