Thursday, November 17, 2016

MongoDB - Configure KMIP backed mongod (with key roration)



[donghua@infrastructure shared]$ python pykmip_server.py
2016-11-17 10:00:11,753 - __main__ - INFO - Starting KMIP server


[donghua@database shared]$ mkdir -p /home/donghua/LAB5

# Configure encryption using a New Key https://docs.mongodb.com/manual/tutorial/configure-encryption/#key-manager

mongod --dbpath /home/donghua/LAB5 --logpath /home/donghua/LAB5/mongo.log --port 31260 --fork --enableEncryption --kmipServerName infrastructure.dbaglobe.com --kmipServerCAFile /home/donghua/shared/certs/ca.pem --kmipClientCertificateFile /home/donghua/shared/certs/client.pem

2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten] MongoDB starting : pid=5896 port=31260 dbpath=/home/donghua/LAB5 64-bit host=database.dbaglobe.com
2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten] db version v3.2.10
2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten] git version: 79d9b3ab5ce20f51c272b4411202710a082d0317
2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016
2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten] allocator: tcmalloc
2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten] modules: enterprise
2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten] build environment:
2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten]     distmod: ubuntu1604
2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten]     distarch: x86_64
2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten]     target_arch: x86_64
2016-11-17T23:00:14.781+0800 I CONTROL  [initandlisten] options: { net: { port: 31260 }, processManagement: { fork: true }, security: { enableEncryption: true, kmip: { clientCertificateFile: "/home/donghua/shared/certs/client.pem", serverCAFile: "/home/donghua/shared/certs/ca.pem", serverName: "infrastructure.dbaglobe.com" } }, storage: { dbPath: "/home/donghua/LAB5" }, systemLog: { destination: "file", path: "/home/donghua/LAB5/mongo.log" } }
2016-11-17T23:00:14.804+0800 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),extensions=[local=(entry=mongo_addWiredTigerEncryptors)],encryption=(name=AES256-CBC,keyid=".system"),
2016-11-17T23:00:14.819+0800 I STORAGE  [initandlisten] Created KMIP key with id: 1
2016-11-17T23:00:14.877+0800 I STORAGE  [initandlisten] Encryption key manager initialized using KMIP key with id: 1.
2016-11-17T23:00:14.878+0800 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/home/donghua/LAB5/diagnostic.data'
2016-11-17T23:00:14.879+0800 I NETWORK  [HostnameCanonicalizationWorker] Starting hostname canonicalization worker
2016-11-17T23:00:14.916+0800 I NETWORK  [initandlisten] waiting for connections on port 31260


# (Optional) Rotate the key https://docs.mongodb.com/manual/tutorial/rotate-encryption-key/

mongod --dbpath /home/donghua/LAB5 --logpath /home/donghua/LAB5/mongo.log --port 31260 --fork --enableEncryption --kmipRotateMasterKey --kmipServerName infrastructure.dbaglobe.com --kmipServerCAFile /home/donghua/shared/certs/ca.pem --kmipClientCertificateFile /home/donghua/shared/certs/client.pem

donghua@database:~/LAB5$ cat /home/donghua/LAB5/mongo.log
2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten] MongoDB starting : pid=5962 port=31260 dbpath=/home/donghua/LAB5 64-bit host=database.dbaglobe.com
2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten] db version v3.2.10
2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten] git version: 79d9b3ab5ce20f51c272b4411202710a082d0317
2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016
2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten] allocator: tcmalloc
2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten] modules: enterprise
2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten] build environment:
2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten]     distmod: ubuntu1604
2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten]     distarch: x86_64
2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten]     target_arch: x86_64
2016-11-17T23:07:05.227+0800 I CONTROL  [initandlisten] options: { net: { port: 31260 }, processManagement: { fork: true }, security: { enableEncryption: true, kmip: { clientCertificateFile: "/home/donghua/shared/certs/client.pem", rotateMasterKey: true, serverCAFile: "/home/donghua/shared/certs/ca.pem", serverName: "infrastructure.dbaglobe.com" } }, storage: { dbPath: "/home/donghua/LAB5" }, systemLog: { destination: "file", path: "/home/donghua/LAB5/mongo.log" } }
2016-11-17T23:07:05.250+0800 I -        [initandlisten] Detected data files in /home/donghua/LAB5 created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2016-11-17T23:07:05.250+0800 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),extensions=[local=(entry=mongo_addWiredTigerEncryptors)],encryption=(name=AES256-CBC,keyid=".system"),
2016-11-17T23:07:05.598+0800 I STORAGE  [initandlisten] Created KMIP key with id: 2
2016-11-17T23:07:05.685+0800 I STORAGE  [initandlisten] Rotated master encryption key from id 1 to id 2.
2016-11-17T23:07:05.685+0800 I CONTROL  [initandlisten] now exiting
2016-11-17T23:07:05.685+0800 I NETWORK  [initandlisten] shutdown: going to close listening sockets...
2016-11-17T23:07:05.685+0800 I NETWORK  [initandlisten] removing socket file: /tmp/mongodb-31260.sock
2016-11-17T23:07:05.685+0800 I NETWORK  [initandlisten] shutdown: going to flush diaglog...
2016-11-17T23:07:05.685+0800 I NETWORK  [initandlisten] shutdown: going to close sockets...
2016-11-17T23:07:05.685+0800 I STORAGE  [initandlisten] WiredTigerKVEngine shutting down
2016-11-17T23:07:05.698+0800 I STORAGE  [initandlisten] shutdown: removing fs lock...
2016-11-17T23:07:05.698+0800 I CONTROL  [initandlisten] dbexit:  rc: 0

# Start MongoDB again

mongod --dbpath /home/donghua/LAB5 --logpath /home/donghua/LAB5/mongo.log --port 31260 --fork --enableEncryption --kmipServerName infrastructure.dbaglobe.com --kmipServerCAFile /home/donghua/shared/certs/ca.pem --kmipClientCertificateFile /home/donghua/shared/certs/client.pem

donghua@database:~/LAB5$ cat /home/donghua/LAB5/mongo.log
2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten] MongoDB starting : pid=6004 port=31260 dbpath=/home/donghua/LAB5 64-bit host=database.dbaglobe.com
2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten] db version v3.2.10
2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten] git version: 79d9b3ab5ce20f51c272b4411202710a082d0317
2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016
2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten] allocator: tcmalloc
2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten] modules: enterprise
2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten] build environment:
2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten]     distmod: ubuntu1604
2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten]     distarch: x86_64
2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten]     target_arch: x86_64
2016-11-17T23:07:55.939+0800 I CONTROL  [initandlisten] options: { net: { port: 31260 }, processManagement: { fork: true }, security: { enableEncryption: true, kmip: { clientCertificateFile: "/home/donghua/shared/certs/client.pem", serverCAFile: "/home/donghua/shared/certs/ca.pem", serverName: "infrastructure.dbaglobe.com" } }, storage: { dbPath: "/home/donghua/LAB5" }, systemLog: { destination: "file", path: "/home/donghua/LAB5/mongo.log" } }
2016-11-17T23:07:55.962+0800 I -        [initandlisten] Detected data files in /home/donghua/LAB5 created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2016-11-17T23:07:55.962+0800 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),extensions=[local=(entry=mongo_addWiredTigerEncryptors)],encryption=(name=AES256-CBC,keyid=".system"),
2016-11-17T23:07:56.151+0800 I STORAGE  [initandlisten] Encryption key manager initialized using KMIP key with id: 2.
2016-11-17T23:07:56.151+0800 I CONTROL  [initandlisten]
2016-11-17T23:07:56.151+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2016-11-17T23:07:56.151+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2016-11-17T23:07:56.151+0800 I CONTROL  [initandlisten]
2016-11-17T23:07:56.151+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2016-11-17T23:07:56.151+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2016-11-17T23:07:56.151+0800 I CONTROL  [initandlisten]
2016-11-17T23:07:56.153+0800 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/home/donghua/LAB5/diagnostic.data'
2016-11-17T23:07:56.153+0800 I NETWORK  [initandlisten] waiting for connections on port 31260
2016-11-17T23:07:56.153+0800 I NETWORK  [HostnameCanonicalizationWorker] Starting hostname canonicalization worker
 
 
MongoDB Enterprise > db.getSisterDB('admin').runCommand({getCmdLineOpts: 1})
{
        "argv" : [
                "mongod",
                "--dbpath",
                "/home/donghua/LAB5",
                "--logpath",
                "/home/donghua/LAB5/mongo.log",
                "--port",
                "31260",
                "--fork",
                "--enableEncryption",
                "--kmipServerName",
                "infrastructure.dbaglobe.com",
                "--kmipServerCAFile",
                "/home/donghua/shared/certs/ca.pem",
                "--kmipClientCertificateFile",
                "/home/donghua/shared/certs/client.pem"
        ],
        "parsed" : {
                "net" : {
                        "port" : 31260
                },
                "processManagement" : {
                        "fork" : true
                },
                "security" : {
                        "enableEncryption" : true,
                        "kmip" : {
                                "clientCertificateFile" : "/home/donghua/shared/certs/client.pem",
                                "serverCAFile" : "/home/donghua/shared/certs/ca.pem",
                                "serverName" : "infrastructure.dbaglobe.com"
                        }
                },
                "storage" : {
                        "dbPath" : "/home/donghua/LAB5"
                },
                "systemLog" : {
                        "destination" : "file",
                        "path" : "/home/donghua/LAB5/mongo.log"
                }
        },
        "ok" : 1
}


=====================================================
[donghua@infrastructure shared]$ sudo pip install PyKMIP==0.4.0

[donghua@infrastructure shared]$ cat pykmip_server.py
#!/usr/bin/python

# this file is a thin wrapper around the PyKMIP server
# which is required for some encrypted storage engine tests

import logging

from kmip.services.kmip_server import KMIPServer

def main():

    logger = logging.getLogger(__name__)

    server = KMIPServer(
        host="infrastructure.dbaglobe.com",
        port=5696,
        keyfile="/home/donghua/shared/certs/server.pem",
        certfile="/home/donghua/shared/certs/server.pem",
        cert_reqs="CERT_REQUIRED",
        ssl_version="PROTOCOL_TLSv1",
        ca_certs="/home/donghua/shared/certs/ca.pem",
        do_handshake_on_connect=True,
        suppress_ragged_eofs=True)

    logger.info("Starting KMIP server")

    try:
        server.serve()
    except Exception as e:
        logger.info('Exception received while serving: {0}'.format(e))
    finally:
        server.close()

    logger.info("Stopping KMIP server")


if __name__ == '__main__':
    main()