Difference between revisions of "Pulumi"
From Christoph's Personal Wiki
(Created page with "'''Pulumi''' is an Infrastructure as Code tool that is similar to Terraform, except that it uses Python instead of HCL. ==Install Pulumi== See [https://www.pulumi.co...") |
(No difference)
|
Latest revision as of 19:11, 1 October 2021
Pulumi is an Infrastructure as Code tool that is similar to Terraform, except that it uses Python instead of HCL.
Install Pulumi
See here for details.
- Install the Pulumi binaries:
$ curl -fsSL https://get.pulumi.com | sh $ pulumi version v3.2.1
AWS
Basic example
- Set up environment:
$ export AWS_ACCESS_KEY_ID=<change me> $ export AWS_SECRET_ACCESS_KEY="<change me>" # NOTE: The following is not needed: #$ cat requirements.txt #$ sudo -H python3 -m pip install -r requirements.txt --ignore-installed PyYAML
- Create a new Pulumi Stack
- Create a new Pulumi Stack:
$ pulumi new aws-python This command will walk you through creating a new Pulumi project. Enter a value or leave blank to accept the (default), and press <ENTER>. Press ^C at any time to quit. project name: (demo) project description: (A minimal AWS Python Pulumi program) Created project 'demo' Please enter your desired stack name. To create a stack in an organization, use the format <org-name>/<stack-name> (e.g. `acmecorp/dev`). stack name: (dev) Created stack 'dev' aws:region: The AWS region to deploy into: (us-east-1) us-west-2 Saved config Creating virtual environment... Finished creating virtual environment Updating pip, setuptools, and wheel in virtual environment... Finished installing dependencies Your new project is ready to go! To perform an initial deployment, run 'pulumi up'
- Run the code:
$ pulumi up
View Live: https://app.pulumi.com/xtof/demo/dev/updates/1
Type Name Status Info
+ pulumi:pulumi:Stack demo-dev created 114 messages
+ └─ aws:s3:Bucket xtof-pulumi-bucket created
Outputs:
bucket_name: "xtof-pulumi-bucket-aaaaaaa"
Resources:
+ 2 created
Duration: 1m11s
real 1m34.351s
user 0m0.723s
sys 0m0.294s
- Print the name of the bucket we just created:
$ pulumi stack output bucket_name xtof-pulumi-bucket-aaaaaaa
- Check that the bucket exists in AWS:
$ aws s3 ls $(pulumi stack output bucket_name) 2021-05-11 15:24:49 70 index.html
- Run the code again:
$ pulumi up
Do you want to perform this update? details
pulumi:pulumi:Stack: (same)
[urn=urn:pulumi:dev::demo::pulumi:pulumi:Stack::demo-dev]
~ aws:s3/bucket:Bucket: (update)
[id=xtof-pulumi-bucket-aaaaaaa]
[urn=urn:pulumi:dev::demo::aws:s3/bucket:Bucket::xtof-pulumi-bucket]
[provider=urn:pulumi:dev::demo::pulumi:providers:aws::default_4_3_0::aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee]
+ website: {
+ indexDocument: "index.html"
}
--outputs:--
+ bucket_endpoint: output<string>
~ aws:s3/bucketObject:BucketObject: (update)
[id=index.html]
[urn=urn:pulumi:dev::demo::aws:s3/bucketObject:BucketObject::index.html]
[provider=urn:pulumi:dev::demo::pulumi:providers:aws::default_4_3_0::aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee]
~ acl : "private" => "public-read"
~ contentType: "binary/octet-stream" => "text/html"
Do you want to perform this update? yes
Updating (dev)
View Live: https://app.pulumi.com/xtof/demo/dev/updates/3
Type Name Status Info
pulumi:pulumi:Stack demo-dev 114 messages
~ ├─ aws:s3:Bucket xtof-pulumi-bucket updated [diff: +website]
~ └─ aws:s3:BucketObject index.html updated [diff: ~acl,contentType]
Outputs:
+ bucket_endpoint: "http://xtof-pulumi-bucket-aaaaaaa.s3-website-us-west-2.amazonaws.com"
bucket_name : "xtof-pulumi-bucket-aaaaaaa"
Resources:
~ 2 updated
1 unchanged
Duration: 8s
- Check the API endpoint:
$ curl http://xtof-pulumi-bucket-aaaaaaa.s3-website-us-west-2.amazonaws.com
<html>
<body>
<h1>Hello, Pulumi!</h1>
</body>
</html>
Azure
- The following Pulumi code will create an Azure Kubernetes Service (AKS) cluster in Azure
- Create a configuration file to store all variables:
$ cat << EOF > Pulumi.dev.yaml
config:
aks:prefix_name: xtof-aks-dev
aks:subscription_id: aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
aks:clientId: ffffffff-gggg-hhhh-iiii-jjjjjjjjjjjj
aks:clientSecret:
secure: AAA...
aks:location: westus2
aks:kubernetes_version: 1.19.11
aks:aks_admin_username: k8sadmin
aks:system_pool_profile:
name: agentpool
count: 2
max_pods: 110
mode: System
node_labels: {}
os_disk_size_gb: 200
os_type: Linux
type: VirtualMachineScaleSets
vm_size: Standard_DS3_v2
aks:user_pool_profile:
name: standardpool
count: 15
max_pods: 110
mode: User
node_labels: {}
os_disk_size_gb: 200
os_disk_type: Managed
os_type: Linux
type: VirtualMachineScaleSets
vm_size: Standard_D3_v2 # https://docs.microsoft.com/en-us/azure/virtual-machines/dv2-dsv2-series
aks:aks_network_profile:
pod_cidr: 10.231.0.0/18
service_cidr: 10.231.64.0/19
dns_service_ip: 10.231.64.10
docker_bridge_cidr: 172.17.0.1/16
aks:subnet_id: "/subscriptions/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee/resourceGroups/xtof-aks-dev-rg/providers/Microsoft.Network/virtualNetworks/xtof-aks-dev-vnet/subnets/xtof-aks-dev-subnet"
aks:private_endpoint_name: "dev.xtof.privatelink.uswest2.azmk8s.io"
azure-native:location: westus2
EOF
- Create a "tags" module:
$ cat << EOF > tags.py
# NOTE: Service returned an error. Status=400 Code="InvalidTagNameCharacters"
# Message="The tag names 'kubernetes.io/cluster/test' have reserved characters
# '<,>,%,&,\\,?,/' or control characters. These characters are only allowed for
# tags that start with the prefix 'hidden, link'."
standard_tags = {
"BusinessValue": "R&D",
"CostCenter": "Foobar",
"Customer": "MySelf",
"Environment": "dev",
"Owner": "xtof",
"CreatedBy": "Pulumi",
"Created": "2021-05-13"
}
EOF
- Create an Azure Resource Group module:
$ cat << EOF > resource_group.py
from pulumi import Config
from pulumi_azure_native import resources
import tags
config = Config()
location = config.get("location")
# SEE: https://www.pulumi.com/docs/reference/pkg/azure-native/resources/resourcegroup/
def create_resource(rg_name):
resource_group = resources.ResourceGroup(
rg_name,
resource_group_name=rg_name,
location=location,
tags={
'CostTech': 'aks',
**tags.standard_tags
}
)
return resource_group
EOF
- Create the "main" module:
$ cat << EOF > __main__.py
"""An Azure RM Python Pulumi program"""
import base64
import pulumi
from pulumi_azure_native import resources, containerservice, storage
import pulumi_azure_native as azure_native
import pulumi_azuread as azuread
import pulumi_random as random
import pulumi_tls as tls
import tags
import resource_group
# Set variables
config = pulumi.Config()
prefix_name = config.get("prefix_name")
subscription_id = config.get("subscription_id")
location = config.get("location")
subnet_id = config.get("subnet_id")
private_endpoint_name = config.get("private_endpoint_name")
system_pool_profile = config.require_object("system_pool_profile")
user_pool_profile = config.require_object("user_pool_profile")
aks_network_profile = config.require_object("aks_network_profile")
resource_group_obj = resource_group.create_resource(prefix_name + "-rg")
# Generate an SSH key
ssh_key = tls.PrivateKey("ssh-key", algorithm="RSA", rsa_bits=4096)
managed_cluster_name = config.get("managedClusterName")
if managed_cluster_name is None:
managed_cluster_name = prefix_name
# Create AKS cluster
# SEE: https://www.pulumi.com/docs/reference/pkg/azure-native/containerservice/managedcluster/
# TODO: https://www.pulumi.com/docs/reference/pkg/azure-native/containerservice/managedcluster/#createupdate-aad-managed-cluster-with-enableazurerbac
# TODO: https://www.pulumi.com/docs/reference/pkg/azure-native/network/azurefirewall/
# TODO: https://www.pulumi.com/docs/reference/pkg/azure-native/containerservice/managedcluster/#managedclusteraadprofile
managed_cluster = containerservice.ManagedCluster(
managed_cluster_name,
resource_group_name=resource_group_obj.name,
addon_profiles={},
agent_pool_profiles=[containerservice.ManagedClusterAgentPoolProfileArgs(
enable_node_public_ip=False,
name=system_pool_profile.get("name"),
count=system_pool_profile.get("count"),
max_pods=system_pool_profile.get("max_pods"),
mode=system_pool_profile.get("mode"),
node_labels=system_pool_profile.get("node_labels"),
os_disk_size_gb=system_pool_profile.get("os_disk_size_gb"),
os_type=system_pool_profile.get("os_type"),
type=system_pool_profile.get("type"),
vm_size=system_pool_profile.get("vm_size"),
vnet_subnet_id=config.get("subnet_id"),
)],
api_server_access_profile=containerservice.ManagedClusterAPIServerAccessProfileArgs(
enable_private_cluster=True,
),
enable_rbac=True,
kubernetes_version=config.get("kubernetes_version"),
linux_profile={
"admin_username": config.get("aks_admin_username"),
"ssh": {
"public_keys": [{
"key_data": ssh_key.public_key_openssh,
}],
},
},
identity=containerservice.ManagedClusterIdentityArgs(
type=containerservice.ResourceIdentityType.SYSTEM_ASSIGNED),
dns_prefix=resource_group_obj.name,
network_profile=containerservice.ContainerServiceNetworkProfileArgs(
network_plugin="azure",
pod_cidr=aks_network_profile.get("pod_cidr"),
service_cidr=aks_network_profile.get("service_cidr"),
docker_bridge_cidr=aks_network_profile.get("docker_bridge_cidr"),
dns_service_ip=aks_network_profile.get("dns_service_ip"),
outbound_type="userDefinedRouting",
),
node_resource_group=f"MC_{managed_cluster_name}_westus",
service_principal_profile={
"client_id": config.get("clientId"),
"secret": config.get("clientSecret")
},
tags={
'CostTech': 'aks',
**tags.standard_tags
}
)
# SEE: https://www.pulumi.com/docs/reference/pkg/azure-native/containerservice/agentpool/#agentpoolmode
user_agent_pool = containerservice.AgentPool(
"userPool",
agent_pool_name=user_pool_profile.get("name"),
count=user_pool_profile.get("count"),
max_pods=user_pool_profile.get("max_pods"),
mode=user_pool_profile.get("mode"),
node_labels=user_pool_profile.get("node_labels"),
enable_node_public_ip=False,
# enable_encryption_at_host=True,
os_type=user_pool_profile.get("os_type"),
os_disk_size_gb=user_pool_profile.get("os_disk_size_gb"),
os_disk_type=user_pool_profile.get("os_disk_type"),
resource_group_name=prefix_name + "-rg",
resource_name_="primerai-aks-dev-aks7f632331",
type=user_pool_profile.get("type"),
vm_size=user_pool_profile.get("vm_size"),
vnet_subnet_id=config.get("subnet_id")
)
# SEE: https://www.pulumi.com/docs/reference/pkg/azure-native/storage/storageaccount/#storageaccountcreate
minio_account = storage.StorageAccount(
"minio",
account_name="aksminiorandomname",
resource_group_name=f"MC_{managed_cluster_name}_westus",
location=location,
# kind="StorageV2",
kind="Storage",
minimum_tls_version="TLS1_2",
sku=storage.SkuArgs(
name="Standard_LRS",
),
network_rule_set=storage.NetworkRuleSetArgs(
bypass="AzureServices",
default_action="Deny",
ip_rules=[],
virtual_network_rules=[storage.VirtualNetworkRuleArgs(
virtual_network_resource_id=config.get("subnet_id"),
)],
),
tags={
'CostTech': 'aks',
**tags.standard_tags
}
)
creds = pulumi.Output.all(resource_group_obj.name, managed_cluster.name).apply(
lambda args:
containerservice.list_managed_cluster_user_credentials(
resource_group_name=args[0],
resource_name=args[1])
)
output_private_ssh_key = pulumi.Output.all(ssh_key.private_key_pem)
# Export kubeconfig
encoded = creds.kubeconfigs[0].value
kubeconfig = encoded.apply(
lambda enc: base64.b64decode(enc).decode())
pulumi.export("kubeconfig", kubeconfig)
pulumi.export("private_ssh_key", output_private_ssh_key)
EOF
