Securing secrets with python and vault

Vault

About Vault

Admit it, you’ve been there before. You’re working on a codebase that requires a password, and you think; “there must be a better way of storing our password than on disk”. One answer to this problem is Vault, from Hashicorp, the makers of popular software like Vagrant, Terraform, and more.

Vault allows you to add and manage secrets, and deliver those secrets to clients that are authenticated. It works by putting your secrets in a storage backend like Mysql or AWS’s IAM.

In this example, we will setup vault with consul as it’s backend. Let’s walk through a sample python app that uses a stored credential. Warning, this guide assumes some familiarity with docker and docker-compose. It should also only be considered a development environment. If you have issues, please leave comments below.

Deploying Vault

First, we’ll deploy a small cluster, consisting of 3 consul container and a vault container, with docker-compose.

$ git clone https://github.com/ryanhartje/containers.git
$ cd containers/consul-vault/
$ docker-compose up -d

As a result, you should see our 4 new containers when you run docker ps. Shortly afterward, consul should be up and running it’s UI, which you should be able to access at:
http://localhost:8500/ui/#/localhost/services

Next, we’ll need to initialize and unlock our vault before we’re able to use it. Be sure to store the keys generated in this next step. If your container restarts, you will need at least 3 of them in order to unlock it again.

$ docker-compose exec vault /vault/vault init
Unseal Key 1: Ew/HlH6gsa0rVxlGGPYAa1Cyrjtrz3quLOoNbR9WpQMB
Unseal Key 2: gkehCe1PdhMdvqaXcpoo3rkSBbfOyJurhYv5daC+NJMC
Unseal Key 3: YBlCM3wGj+0O/nf8uFdqQNNFPaGnbaBzSyJ4jM6V8bsD
Unseal Key 4: keNOmgduRXl++mXpNeIDk2BvCuz51GgUgfrNbfyC8c0E
Unseal Key 5: c72toJYnvIdturSC/y9BDQo4MvqQcVPMT1NMlJKpNOUF
Initial Root Token: f9d963f6-0766-efee-9272-13602d329aea

Vault initialized with 5 keys and a key threshold of 3. Please
securely distribute the above keys. When the Vault is re-sealed,
restarted, or stopped, you must provide at least 3 of these keys
to unseal it again.

Vault does not store the master key. Without at least 3 keys,
your Vault will remain permanently sealed.

Now it’s time to unseal Vault. You’ll need to copy and paste 3 of your keys during this process. For this reason, we’ll need to use an interactive shell for this portion. We will then also authenticate initially to get access to admin functions.

$ docker exec -ti vault /bin/bash
root@vault:/# /vault/vault unseal
Key (will be hidden):
Sealed: true
Key Shares: 5
Key Threshold: 3
Unseal Progress: 1
root@vault:/# /vault/vault unseal
Key (will be hidden):
Sealed: true
Key Shares: 5
Key Threshold: 3
Unseal Progress: 2
root@vault:/# /vault/vault unseal
Key (will be hidden):
Sealed: false
Key Shares: 5
Key Threshold: 3
Unseal Progress: 0

root@vault:/# vault auth
Token (will be hidden):
Successfully authenticated! You are now logged in.
token: f9d963f6-0766-efee-9272-13602d329aea
token_duration: 0
token_policies: [root]

Now we are ready to go. We need to add a credential to it, and make sure we get the right thing when we ask for it. As a final measure, we must generate an auth token for our python app to use. Ideally, we’d use an authentication backend, like LDAP or EC2, however for this demo, we’ll issue one.

$ docker exec vault /vault/vault token-create
Key Value
--- -----
token 3340a910-0d87-bb50-0385-a7a3e387f2a8
token_accessor 60aa9e57-62a8-7657-8831-8f9f043c9c55
token_duration 0s
token_renewable false
token_policies [root]
$ docker exec vault /vault/vault write /secret/hello world=3340a910-0d87-bb50-0385-a7a3e387f2a8
Success! Data written to: secret/hello

$ docker exec vault /vault/vault read /secret/hello
Key Value
--- -----
refresh_interval 768h0m0s
world 3340a910-0d87-bb50-0385-a7a3e387f2a8

Python and Vault

Now we can use Python to access secrets that we are storing inside of Vault. In order to do so, we’ll first have to install hvac via pip or your favorite package manager.

$ pip install --user -r requirements.txt
Collecting hvac (from -r requirements.txt (line 1))
 Downloading hvac-0.2.17-py2.py3-none-any.whl
Requirement already satisfied: requests>=2.7.0 in /Library/Python/2.7/site-packages (from hvac->-r requirements.txt (line 1))
Installing collected packages: hvac

In our example, let’s follow some good practices and configure our app via environment variables. To start, let’s put the token we generated above (not the root token) into an environment variable called VAULT_TOKEN. Let’s put our URL in there as well.

$ export VAULT_TOKEN=3340a910-0d87-bb50-0385-a7a3e387f2a8 
$ export VAULT_URL=http://localhost:8200

In this configuration, we disabled TLS for simplicity. This is not recommended for production use. Other shortcuts were taken to expose the service on port 8200 so we can run our script locally.

In our script, we’ll setup an hvac client, connect to localhost:8200, pass in our token from the environment variable, and finally read, write and delete our own secrets from our script.

Here’s a short script that allows us to do just that:

import os

import hvac

client = hvac.Client()
client = hvac.Client(
 url=os.environ['VAULT_URL'],
 token=os.environ['VAULT_TOKEN']
)
client.write('secret/snakes', type='pythons', lease='1h')
print(client.read('secret/snakes'))

If we run our simple script, we’ve successfully written and retrieved a secret from vault.

$ python simple_example.py
{u'lease_id': u'', u'warnings': None, u'wrap_info': None, u'auth': None, u'lease_duration': 3600, u'request_id': u'c383e53e-43da-d491-6c20-b0f5f7e4a33a', u'data': {u'type': u'pythons', u'lease': u'1h'}, u'renewable': False}

I hope this a helpful walkthrough on how to get started using Vault with Python. If you have any questions or comments, please leave them below. If you or your team want help integrating vault and your applications, let’s talk.