Admin Partitions with HCP Consul and Amazon Elastic Container Service
Enterprise Only
Consul admin partitions is a feature of Consul Enterprise. The enterprise license is provided to you when HCP Consul deploys into your account. This tutorial deploys billable resources to your HashiCorp Cloud and AWS accounts.
Consul admin partitions let organizations define administrative boundaries for services using Consul. This helps organizations manage Consul as a global installation, with services hosted across many teams and business units. Teams benefit by managing and customizing their Consul environment in context to their workloads, without creating impact to other teams, or other Consul environments.
The following diagram shows the potential of Admin partitions within a single region. On the left, an organization works with many teams to support many individual, bespoke Consul clusters. On the right, the Consul cluster operates as a single unified server, with workloads registering into it via one, or many Consul client tenant clusters.
With admin partitions, teams do not need to share the responsibility of maintaining individual server clusters with the organization. These teams focus on contextually-relevant tasks and maintenance directly related to the business value they are delivering, such as service discovery and network automation. Organizations utilize the Consul server cluster as a unified control plane for this fleet of Consul client tenant instances.
Admin partitions creates a mechanism for teams to deploy and share services across their organization, and across other Consul client clusters. This automates the provisioning and registration of trusted Consul clients to a cluster. Admin partitions increases the velocity of a team's ability to deliver business value, eliminating the operational overhead of teams requesting resources from an organization. Admin partitions gives teams autonomy and agency, while the organization maintains control and support of the global Consul installation inside the organization.
In this tutorial, you will deploy HashiCups across two admin partitions, using HCP Consul and two Amazon Elastic Container Service (ECS) clusters, to Consul service mesh. Each ECS cluster is assigned to an admin partition, with HashiCups services hosted in both partitions. Configure HashiCups across partitions using Consul service mesh configuration entries to create the HashiCups deployment. Finish by verifying the services in Consul, then confirming the operation of HashiCups in your web browser.
Prerequisites
- An Amazon Web Services (AWS) account with permission to deploy Amazon Elastic Container Service resources.
- Access to AWS credentials to deploy resources via Terraform.
- HashiCorp Cloud (HCP) Service Principal Credentials. Learn how to create Service Principals by reading the HCP Docs Service Principals documentation page.
Configure required project resources
Begin by cloning the repository.
Navigate into the repository folder.
Fetch the tags from the remote git server to checkout the git tag for this tutorial.
Navigate into the project folder for this tutorial.
Set your HCP service principal credentials as environment variables.
Deploy required project resources
This tutorial begins by deploying an HCP Cluster, and two Amazon ECS clusters to deploy HashiCups across two ECS clusters. You will build upon this code, using Consul to set the admin partition for each tenant cluster. One partition (default) for private, internal services, another partition (part2) for public-facing services accessed via the internet. The default partition is assigned to HCP Consul, spanning HCP Consul, and one ECS Cluster, clust1. The part2 partition is assigned to ECS cluster, clust2. The following diagram will help you familiarize yourself with this architectural setup.
Note
Using Consul on ECS, admin partitions are assigned to individual ECS Clusters. An ECS cluster can belong to
the default
partition, but cannot be assigned to other partitions assigned to other Amazon ECS clusters.
Initialize the terraform project.
Deploy the initial resources for this tutorial to your AWS and HCP accounts, consisting of your HCP Consul Cluster,
and two Amazon ECS clusters. Use terraform apply
to deploy, which presents a confirmation screen to deploy
the resources. Type “yes” to confirm the deployment of these resources.
The ECS mesh-task terraform submodule creates your application's ECS task definition, including Consul-specific configuration to register the task definition as both a Consul node, and service.
Create HashiCups infrastructure
Your HCP and AWS accounts currently include an HCP Consul cluster, and two Amazon ECS clusters. Continue on, building the Terraform code for the HashiCups application and surrounding infrastructure.
Create a file for the ACL controllers in the current project folder.
Each ECS cluster and Consul partition uses an ACL controller to manage task access to HCP Consul. Create ACL Controllers for each ECS cluster, noting the highlights which enable partitions, and assigns each ACL controller to an ECS cluster and partition.
Next, create the private and public task definitions for HashiCups using the mesh-task
submodule.
The private task definitions in the first submodule block represent HashiCups services assigned to the default
admin partition on ECS cluster, clust1. The consul_partition
parameter for each mesh-task
represent the
admin partitions to which each group of tasks in mesh-task
is being assigned. local.admin_partitions.one
represents the default
partitions on Amazon ECS cluster, clust1. local.admin_partitions.two
represents the part2
partition on Amazon ECS cluster, clust2.
The tasks in each task definition group (public, and private) are created using by using the
terraform-aws-consul-ecs mesh-task
submodule, once for public, and once for private. To assign a task definition to an admin partition,
the submodule uses parameters for the partition and namespace to assign the tasks to specified values.
Before creating the tutorial's task definitions, review the code sample below to observe the parameters in context of the submodule. To learn more, read the mesh-task usage docs on consul.io.
1 2 3 4 5 6 7 8 9 1011121314
Create a file for these task definitions, in the current project folder.
Paste the following code into hashicups-tasks.tf
Next, create data resources for each ECS task definition created from the mesh-task
submodule. The data resources
use metadata from their underlying ECS task definitions, mappingmesh-task
task definitions to AWS ECS Service
resources.
Insert the code below at the end of data.tf
, in the current project folder.
Create the second admin partition
HCP Consul generates the default admin partition at the time of installation. Subsequent partitions are created by referencing a new partition name in a service configuration file, or by creating the partition resource via Consul cluster configuration. You will create the partition via Consul cluster configuration using Terraform. This partition, part2, consists of public-facing HashiCups services on ECS cluster, clust2.
Create a file for the admin partition in the current project folder.
Paste the code below, creating the part2
admin partition.
Create exported services
HashiCups services span two Amazon ECS Clusters, in different admin partitions. Consul's Exported Services feature defines which services can communicate outside its admin partition.
Create exported service entries for product-api and public-api. These two services communicate with each other in HashiCups.
Create a file for the exported services, in the current project folder.
Insert the following code for product-api and public-api.
Create service defaults
The product-api uses a service defaults configuration in Consul Service Mesh to declare its service protocol as a default global value.
Create a file for the service defaults, in the current project folder.
Paste the code below into consul-service_defaults.tf
.
Create service intentions
Service intentions permit access between source and destination services in the Consul service mesh. Create service intentions for services which communicate with each other in the HashiCups application.
Create a file for the service intentions, in the current project folder.
Paste the code below, into consul-service_intentions.tf
.
Create HashiCups Amazon ECS services
To deploy HashiCups, each mesh-task
definition operates as an Amazon ECS Service. The aws_ecs_service
terraform
resource creates the deployment for the task definition. When the ECS service finishes deploying, each task definition
is represented in ECS as an active task. In HCP Consul each active task is represented as a service in Consul service
mesh, and as a node in the Consul cluster.
Create a file for the ECS Services, in the current project folder.
Place the code into the file.
Create an outputs file, in the current project folder.
You will create ouputs from the deployed resources to log in to HCP Consul, and observe the HashiCups application in a web browser.
Place the following code in outputs.tf
.
Using terraform apply
, deploy the HashiCups application and related configuration.
Validate services
Using terraform output
, retrieve the login token in your shell. Copy the HCP Consul URL in thevalue.consul_ui_address
stanza of the json output.
1 2 3 4 5 6 7 8 9 10111213141516171819202122232425262728
Navigate to the Consul UI URL in your browser. Log in with the token.
After logging in, click the “Admin Partition” dropdown menu in the top-left corner, selecting an admin partition to observe services for the selected partition.
Click on Intentions to observe the Service Intentions in each partition.
Visit HashiCups application
Next, navigate to the HashiCups URL in your browser. Retrieve the URL with terraform output
. Copy the value in the
outputs_not_sensitive.value.hashicups_url
stanza of your json output.
When the page loads, the HashiCups application renders on-screen, with a collection of beverages to choose from, for (fictional) purchase. This confirms HashiCups services are communicating across partitions, across Amazon ECS clusters. This concludes the tutorial.
Clean up
Bring down the infrastructure using terraform destroy
.
The clean-up process takes up to 20 minutes.
Next steps
In this tutorial, you deployed HCP Consul, two Amazon ECS clusters, HashiCups tasks as services deployed to Consul on ECS. These clusters comprised of services deployed across admin partitions, in individual Amazon ECS clusters. To learn more, take the following tutorials and read the docs to learn more about Amazon ECS at HashiCorp, and Consul admin partitions.
- Take the Deploy an application to Amazon Elastic Container Service course on Learn to learn how to use HashiCorp Waypoint with ECS.
- Deploy a Vault agent to Amazon ECS with the Vault Agent with Amazon Elastic Container Service course on Learn.
- Read the Admin Partitions docs on consul.io to learn more about the functionality Consul Admin Partitions.