Hashicorp Vault

Introduction to Vault

Vault is a secret management solution that manages and protects sentitive data. Vault provides a central and single source of details for both humans and machines. Vault also provides a complete lifecycle management of these secrets. The main problem that vault set out to mitigate is secret sprawl by centralizing the location to store these secrets by securrly storing any secret. Vault also provides governance for access to these secrets with access management and audit trails for compliance. Vault supports different kinds of secrets which includes usernames with passwords, certificates, API keys and certificates etc.

Vault has multiple ways to interact with vault. CLI, APIs(For automations and machines), UI(for interactive users) are available for any users or machiens to access vault and retrive the secrets. After the authentication, Vault issues an authorized token that allows us access to the secrets in the vault based on the policies that are associated with the user. This token is a limited time token that can be used anywhere within the vault before it expires.

Now, there are different types of authentication methods which includes

1. Username and Password
2. RoleID & Secret ID
3. TLS Certificate
4. Integrated Cloud Credentials

When we want to request some data from a particular path, the user is going to use the issued token and sends an API request to the vault core, which will verify if the token is valid, not expired and has the permission on the requested path to retrieve the data requested.

Benifits of leveraging Vault:

1. Consolidate all secrets and sensitive data in a single place
2. Store long lived static secrets
3. Dynamically generate keys upon request
4. Fully featured with API
5. Identity based access across clouds and systems
6. Encryption as a service
7. Root or Intermediate Certificates Authority.

Vault Components

The following are the basic components that vault is made up of. These components together make up the vault and its functionality.

1. Storage Backend
2. Secrets Engine
3. Authentication Mechanisms
4. Audit Devices

Storage Backends

• This configures a location where the data of vault data. Any data within the vault is cofigured to store in this storage backend.
• Storage is defined in the main vault configuration file with desired parameters
• All data is encrypted in transit(TLS) and at rest using AES-256
• Not all storage backends are created equally. For example S3 should be backed up manually. Some backends support HA while others have better management and data protection
• There is only one storage backend per vault cluster

Secrets Engine

• This is main core functionality of the vault. This can create, store and encrypt data
• This components is responsible for managing secrets in the vault.
• Many secret engines can be enabled and used as needed. We can even have multiple secret engines of the same type.
• Many secret engines connect to other services to generate dynamic credentials on-demand
• secret engines are enabled and isolated at a path. All interaction are done directly with the path itself that the user has access to based on the policy.

Authentication Methods

• This component is responsible to perform authentication and manage identities. With this, we can assign identities and policies to a user.
• Multiple authentication methods can be enabled depending on use case. This includes OIDC, identity based etc.
• Once authenticated, a client token is issues and can be used to interact with the secrets engine or any other request that needs to be made.
• Each of these token is associated with a policy (can be multiple) that specifies the level of access the token has the requests that it can make. The token is also short-lived with a TTL configured.
• With Vault, tokens are a default authentication method. All auth request main goal is to get this token, which can later be used for all the requests.

Audit Devices

• Vault keeps a details log of all requests and responses, which is where this components comes in.
• Audit log is formatted in JSON and is hashed before logging.
• Vault supports (and is recommanded) to have multiple Audit devices enabled. If vault is not able to write a log to its endpoint, then the request would be successfully in getting a response from vault as Vault prioritizes safety over availablity.

Vault Architecture

The Architecture starts with HTTP/S API and ends with a storage backend to store everything that goes into the vault. Everything in between these two is protected via a barries. To move through this cryptography barries, we would need a token. Storage backend is not included in this barrier because inherently, vault doesn’t trust the storage backend as it can be setup any way the user deems fit.

Everything in vault is path based. All components of the vault are mounted at a specific path and the path directs the requests made to the requested component based on the path.

The path available are dependent on the feature enabled in the Vault like secrets engine, auth methods etc. System backend is the default backend to the vault that is mounted at the path \sys. Vault components can be enabled at any paths that we like using the -path CLI flag. Now, each component does have a default path that we can use if we don’t want to customize it.

Vault has a few path that are pre-allocated and cannot be used, changed or deleted. For example,

1. auth/ – Endpoint for auth method configurations
2. cubbyhole/ – Endpoint used by Cubbyhole secrets engine
3. identity/ – Endpoint for configuring Vault Identity (like entities and groups)
4. secret/ – Endpoint used by Key/Value v2 secret engine (if running in dev mode)
5. sys/ – As aforementioned, system endpoint for configuring Vault.

Seal vs Unseal

When a vault starts it is in a sealed state, meaning it knows where the data is and how to access the data but it doesn’t have an way to decrypt that data. Almost no operations are possible when vault is in a sealed state other than status check and unsealing.

Unsealing Vault means that a node can reconstruct the master key in order ot decrypt the encryption key and ultimately read the data. After unsealing the encryption key is stored in memory which can be used to read and write data. Every single node in the node will have its own key. Sealing Vault means that the vault throws away the encryption key and requires another unseal to perform any further actions. By default, Vault starts in sealed state (Can also be done using CLI, UI and API Keys)

There are multiple options for sealing and unsealing the data, which includes

1. Key Sharding (Using Sharmir encryption)
2. Cloud Auto Unseal
3. Transit Auto Unseal

Note:

Shamir's Secret Sharding allows us to divide the secret or a sensitive data into multiple smaller keys. When storing the data, there are couple of options we can make the data highly available but the trade off being we will have to maintain multiple copies of the same data and none of them should be leaked or we can have a single secure key but it is not highly available and is not easily available. These are just the trade off with each type of deployment.

With Shamir’s secret sharing, we can split the data into multiple piece and will lets us set a threshold. Now, if we have the thershold number of pieces of data, then we can learn what the secret is. If not, we will not be able to assemble what the data/secret is. With this, we have some wigggle room for data that can be compromised without have to compromise the secret.