Setting up Vault Server as Google Cloud Storage as Storage Backend

Hello Folks !

Recently I worked on task to isolate the passwords, database credentials from properties file and store it inside the vault.
This post explain how to install and setup the vault server, store data as Key-value pair and usage of this vault data in spring boot application.
Vault comes with many secrete engines (and so authentication methods) and storage options.

In this I’m going to use KV engine which is simple key-value based store mechanism. but insted of storing KV in file/memory I have used GCP Storage as backent storage.
GCP storage also provide High Avaiability. Even if you change your vault server, you don’t have to change ur root key or unseal keys.

So lets start with downloading and installing the vault.

Download vault from https://www.vaultproject.io/downloads.html. I have downloaded the linux version. Vault runs as a single binary named ‘vault‘.

$> sudo wget https://releases.hashicorp.com/vault/1.3.1/vault_1.3.1_linux_amd64.zip
$> sudo unzip vault_1.3.1_linux_amd64.zip
$> sudo cp vault /usr/local/bin/
$> vault -autocomplete-install 
$> complete -C /usr/local/bin/vault vault
$> vault -version or vault version -- to verify the vault installation

Give Vault the ability to use the mlock syscall without running the process as root. The mlock syscall prevents memory from being swapped to disk.

$> sudo setcap cap_ipc_lock=+ep /usr/local/bin/vault

Now create a unique, non-privileged system user to run Vault.

$> sudo useradd --system --home /etc/vault.d --shell /bin/false vault

Next step is to setup vault configuearion i.e. config.json or config.hcl

$> sudo mkdir /etc/vault
$> sudo vi /etc/vault/config.json -- copy paste following contents in config.json

{
"listener": [{
"tcp": {
"address" : "0.0.0.0:8200",
"tls_disable" : 1
}
}],
"api_addr": "http://<vault-server-machine-ip>:8200",
"storage": {
  "gcs": {
  "bucket" : "vault-bucket-dev",
  "ha_enabled" : "true"
  }
},
"max_lease_ttl": "10h",
"default_lease_ttl": "10h",
"ui":true,
"bindings": [
              {
                "role": "roles/storage.objectAdmin",
                "members": [
                            "serviceAccount:sa-vault-admin@<Your-GCP-Email_account>.gserviceaccount.com"
                           ]
              }
        ]
}

Lets monitor the vault server by creating a linux service as below

$> sudo vi /etc/systemd/system/vault.service --copy the following contents in file

[Unit]
Description=vault service
Requires=network-online.target
After=network-online.target
ConditionFileNotEmpty=/etc/vault/config.json
 
[Service]
User=vault
Group=vault
EnvironmentFile=-/etc/sysconfig/vault
Environment=GOMAXPROCS=2
Restart=on-failure
ExecStart=/usr/local/bin/vault server -config=/etc/vault/config.json
StandardOutput=/var/log/vault/output.log
StandardError=/var/log/vault/error.log
LimitMEMLOCK=infinity
ExecReload=/bin/kill -HUP $MAINPID
KillSignal=SIGTERM
 
[Install]
WantedBy=multi-user.target

Enable and start the vault service as below-

$> sudo systemctl enable vault
$> sudo systemctl start vault
$> sudo systemctl status vault

in case of error while starting vault server, perform the below steps-

Now Login as root and Export VAULT_ADDR environment variable, don’t forget to add this to ~/.bashrc file. Change the IP to you vault server public/private IP.

$> export VAULT_ADDR=http://10.192.246.27:8200
$> echo "export VAULT_ADDR=http://10.192.246.27:8200" >> ~/.bashrc

Now check the vault server status as below
$> vault status

if You face the error like vault not initialzed – means By default vault is initialized to false and sealed as true. Lets initialized vault as below

$> vault operator init > /etc/vault/init.file
$> vault status

Now vault is initiated but sealed. Open the init file to get the unseal and root tokens. These tokens can be used to unseal the vault web UI during the first login.

$>cat /etc/vault/init.file

the init.file contains , 5 unseal keys and 1 root token. distribute this at safe place. Its been used to unseal the vault in case of restart, reboot of system.
Unseal vault using unseal command as below by using any 3 keys

$> vault operator unseal 1KEKrBg3XwccM7Hs8HH4ixmlL6GrJqWTTK8atXX6J8TI
$> vault operator unseal scE27sOiPMFfnlcccINWRwBON5nO7n2ngbps7XXL3QCM
$> vault operator unseal zsqt9wROuy/V2CCVjH5v9u6LJtOvelua2DewUWXXXlxL

The ui=true setting from Config.json is set to enable the Vault Enterprise UI dashboard for Vault.

Using UI , one can add the screte path and key-value pair data. ( note in UI select KV as engine ) Vault dashboard acceesible by URL http://<ur-vault-server-IP-:8200/ui

Login to the UI using token, use the root key value as token. Through UI create a key-value pair secret type. ( select KV as secrete engine), you have to specify the PATH name and key-value pairs. Access the keys as below where gs-vault-config and MQExpoerterService is path name I given-

$> curl  -H "X-Vault-Token: s.sf7LBLXXXXXXXXXXXXXXXPD6" http://<vault-server-ip>:8200/v1/secret/gs-vault-config
$> curl  -H "X-Vault-Token: s.sf7LBLXXXXXXXXXXXXXXXPD6"     http://<vault-server-ip>:8200/v1/secret/MQExporterService

IF you select version as V2 the key access path conatins the data in folder hierarchy.

In next post, I will show how to access vault data i.e. the secrets through Spring Boot Application.

Thanks !!!