HashiCorp Vault¶
The following always refers to a Vault setup that has been deployed via the official HashiCorp Vault helm chart.
Migrate von shamir to an auto-unseal mechansim¶
While migration from shamir to an auto-unseal mechanism it could be, that you see the following error message in the logs:
[WARN] core: entering seal migration mode; Vault will not automatically unseal even if using an autoseal: from_barrier_type=shamir to_barrier_type=azurekeyvault
This can be fixed by running the following command three times:
vault operator unseal -migrate
Migration from File to Raft Storage Engine¶
Some sources: - https://developer.hashicorp.com/vault/tutorials/raft/raft-storage - https://developer.hashicorp.com/vault/tutorials/raft/storage-migration-checklist - https://developer.hashicorp.com/vault/tutorials/raft/raft-migration#migrate-vault-storage
The following guide assumes that you are using Vault in the standalone mode/with a single instance.
Your current config should already contain something like the following. You may already be using the standalone version, where this is the default storage engine. Keep this configuration for now, we will update it in the second step.
storage "file" {
path = "/vault/data"
}
1. Migrate to raft¶
Login to the machine, where your single instance vault is currently running. E.g. if it's running in K8s do the following:
kubectl exec -ti vault-0 -- sh
Depending on your setup, you may have a read-only filesystem. If so, you will need to enter the path where the PVC is mounted (first line in the code block below).
The next step is to create a new file with the migration config. There are
examples of this in the internet, which also include a node_id in the
source_destination block. This is not necessary, as Vault will assign a
random UUID if you don't set the node_id at all. I also got an error when
trying to set the node_id. The paths need to be adjusted to your current
configuration (see the code block at the top of this
migration guide).
cd /vault
cat <<EOF > migrate.hcl
storage_source "file" {
path = "/vault/data/"
}
storage_destination "raft" {
path = "/vault/data/"
}
cluster_addr = "https://vault-0.vault-internal:8201"
EOF
After creating the file we need to execute the migration:
vault operator migrate -config=migrate.hcl
After the successfull execution of this command, you can close the remote shell into the container.
2. Adjust the config¶
Now you need to adjust your Vault configuration. Either you simply switch directly to the HA mode, or you need to replace your config like as pointed out in the diff below:
- storage "file" {
+ storage "raft" {
path = "/vault/data"
}
Troubleshooting¶
Cancel Migration Process¶
If you cancel the migration process e.g. by hitting Ctrl + C and you try to restart it, you will get the following error message:
Error migrating: error mounting 'storage_destination': could not bootstrap clustered storage: error bootstrapping cluster: cluster already has state
High-Availablity (HA) Mode¶
Please condsider first to migrate to the raft storage engine.
Adding a new node¶
If you are using the official helm chart to deploy HashiCorp Vault it's
quite easy to spin up a new instance. All you need to do is increase the field
server.ha.replicas in your values.yaml to the number of instances you want
to have.
The tricky part is adding the nodes to the cluster. You need to manually run
the following command for each new node (replace X with the specific number
of the node according to the k8s pod name):
kubectl exec -ti vault-X -- vault operator raft join http://vault-0.vault-internal:8200
If you haven't configured auto-unseal, you will also need to manually unseal all new nodes. To do this, run the following command three times for each new node:
shell kubectl exec -ti vault-X -- vault operator unseal