Pulumi

From Christoph's Personal Wiki
Revision as of 19:11, 1 October 2021 by Christoph (Talk | contribs) (Created page with "'''Pulumi''' is an Infrastructure as Code tool that is similar to Terraform, except that it uses Python instead of HCL. ==Install Pulumi== See [https://www.pulumi.co...")

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Pulumi is an Infrastructure as Code tool that is similar to Terraform, except that it uses Python instead of HCL.

Install Pulumi

See here for details.

  • Install the Pulumi binaries:
$ curl -fsSL https://get.pulumi.com | sh
$ pulumi version
v3.2.1

AWS

Basic example

  • Set up environment:
$ export AWS_ACCESS_KEY_ID=<change me>
$ export AWS_SECRET_ACCESS_KEY="<change me>"

# NOTE: The following is not needed:
#$ cat requirements.txt
#$ sudo -H python3 -m pip install -r requirements.txt --ignore-installed PyYAML
Create a new Pulumi Stack
  • Create a new Pulumi Stack:
$ pulumi new aws-python
This command will walk you through creating a new Pulumi project.

Enter a value or leave blank to accept the (default), and press <ENTER>.
Press ^C at any time to quit.

project name: (demo) 
project description: (A minimal AWS Python Pulumi program) 
Created project 'demo'

Please enter your desired stack name.
To create a stack in an organization, use the format <org-name>/<stack-name> (e.g. `acmecorp/dev`).
stack name: (dev) 
Created stack 'dev'

aws:region: The AWS region to deploy into: (us-east-1) us-west-2
Saved config

Creating virtual environment...
Finished creating virtual environment
Updating pip, setuptools, and wheel in virtual environment...

Finished installing dependencies
Your new project is ready to go! 

To perform an initial deployment, run 'pulumi up'
  • Run the code:
$ pulumi up

View Live: https://app.pulumi.com/xtof/demo/dev/updates/1

     Type                 Name                Status      Info
 +   pulumi:pulumi:Stack  demo-dev            created     114 messages
 +   └─ aws:s3:Bucket     xtof-pulumi-bucket  created

Outputs:
    bucket_name: "xtof-pulumi-bucket-aaaaaaa"

Resources:
    + 2 created

Duration: 1m11s


real	1m34.351s
user	0m0.723s
sys	0m0.294s
  • Print the name of the bucket we just created:
$ pulumi stack output bucket_name
xtof-pulumi-bucket-aaaaaaa
  • Check that the bucket exists in AWS:
$ aws s3 ls $(pulumi stack output bucket_name)
2021-05-11 15:24:49         70 index.html
  • Run the code again:
$ pulumi up

Do you want to perform this update? details
  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:dev::demo::pulumi:pulumi:Stack::demo-dev]
    ~ aws:s3/bucket:Bucket: (update)
        [id=xtof-pulumi-bucket-aaaaaaa]
        [urn=urn:pulumi:dev::demo::aws:s3/bucket:Bucket::xtof-pulumi-bucket]
        [provider=urn:pulumi:dev::demo::pulumi:providers:aws::default_4_3_0::aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee]
      + website: {
          + indexDocument: "index.html"
        }
    --outputs:--
  + bucket_endpoint: output<string>
    ~ aws:s3/bucketObject:BucketObject: (update)
        [id=index.html]
        [urn=urn:pulumi:dev::demo::aws:s3/bucketObject:BucketObject::index.html]
        [provider=urn:pulumi:dev::demo::pulumi:providers:aws::default_4_3_0::aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee]
      ~ acl        : "private" => "public-read"
      ~ contentType: "binary/octet-stream" => "text/html"

Do you want to perform this update? yes
Updating (dev)

View Live: https://app.pulumi.com/xtof/demo/dev/updates/3

     Type                    Name                Status      Info
     pulumi:pulumi:Stack     demo-dev                        114 messages
 ~   ├─ aws:s3:Bucket        xtof-pulumi-bucket  updated     [diff: +website]
 ~   └─ aws:s3:BucketObject  index.html          updated     [diff: ~acl,contentType]

Outputs:
  + bucket_endpoint: "http://xtof-pulumi-bucket-aaaaaaa.s3-website-us-west-2.amazonaws.com"
    bucket_name    : "xtof-pulumi-bucket-aaaaaaa"

Resources:
    ~ 2 updated
    1 unchanged

Duration: 8s
  • Check the API endpoint:
$ curl http://xtof-pulumi-bucket-aaaaaaa.s3-website-us-west-2.amazonaws.com
<html>
    <body>
        <h1>Hello, Pulumi!</h1>
    </body>
</html>

Azure

The following Pulumi code will create an Azure Kubernetes Service (AKS) cluster in Azure
  • Create a configuration file to store all variables:
$ cat << EOF > Pulumi.dev.yaml
config:
  aks:prefix_name: xtof-aks-dev
  aks:subscription_id: aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
  aks:clientId: ffffffff-gggg-hhhh-iiii-jjjjjjjjjjjj
  aks:clientSecret:
    secure: AAA...
  aks:location: westus2
  aks:kubernetes_version: 1.19.11
  aks:aks_admin_username: k8sadmin
  aks:system_pool_profile:
    name: agentpool
    count: 2
    max_pods: 110
    mode: System
    node_labels: {}
    os_disk_size_gb: 200
    os_type: Linux
    type: VirtualMachineScaleSets
    vm_size: Standard_DS3_v2
  aks:user_pool_profile:
    name: standardpool
    count: 15
    max_pods: 110
    mode: User
    node_labels: {}
    os_disk_size_gb: 200
    os_disk_type: Managed
    os_type: Linux
    type: VirtualMachineScaleSets
    vm_size: Standard_D3_v2 # https://docs.microsoft.com/en-us/azure/virtual-machines/dv2-dsv2-series
  aks:aks_network_profile:
    pod_cidr: 10.231.0.0/18
    service_cidr: 10.231.64.0/19
    dns_service_ip: 10.231.64.10
    docker_bridge_cidr: 172.17.0.1/16
  aks:subnet_id: "/subscriptions/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee/resourceGroups/xtof-aks-dev-rg/providers/Microsoft.Network/virtualNetworks/xtof-aks-dev-vnet/subnets/xtof-aks-dev-subnet"
  aks:private_endpoint_name: "dev.xtof.privatelink.uswest2.azmk8s.io"
  azure-native:location: westus2
EOF
  • Create a "tags" module:
$ cat << EOF > tags.py
# NOTE: Service returned an error. Status=400 Code="InvalidTagNameCharacters"
# Message="The tag names 'kubernetes.io/cluster/test' have reserved characters
# '<,>,%,&,\\,?,/' or control characters. These characters are only allowed for
# tags that start with the prefix 'hidden, link'."

standard_tags = {
    "BusinessValue": "R&D",
    "CostCenter": "Foobar",
    "Customer": "MySelf",
    "Environment": "dev",
    "Owner": "xtof",
    "CreatedBy": "Pulumi",
    "Created": "2021-05-13"
}
EOF
  • Create an Azure Resource Group module:
$ cat << EOF > resource_group.py
from pulumi import Config
from pulumi_azure_native import resources
import tags

config = Config()
location = config.get("location")

# SEE: https://www.pulumi.com/docs/reference/pkg/azure-native/resources/resourcegroup/
def create_resource(rg_name):
    resource_group = resources.ResourceGroup(
        rg_name,
        resource_group_name=rg_name,
        location=location,
        tags={
            'CostTech': 'aks',
            **tags.standard_tags
        }
    )

    return resource_group
EOF
  • Create the "main" module:
$ cat << EOF > __main__.py
"""An Azure RM Python Pulumi program"""

import base64
import pulumi
from pulumi_azure_native import resources, containerservice, storage
import pulumi_azure_native as azure_native
import pulumi_azuread as azuread
import pulumi_random as random
import pulumi_tls as tls
import tags
import resource_group

# Set variables
config = pulumi.Config()
prefix_name = config.get("prefix_name")
subscription_id = config.get("subscription_id")
location = config.get("location")
subnet_id = config.get("subnet_id")
private_endpoint_name = config.get("private_endpoint_name")
system_pool_profile = config.require_object("system_pool_profile")
user_pool_profile = config.require_object("user_pool_profile")
aks_network_profile = config.require_object("aks_network_profile")

resource_group_obj = resource_group.create_resource(prefix_name + "-rg")

# Generate an SSH key
ssh_key = tls.PrivateKey("ssh-key", algorithm="RSA", rsa_bits=4096)

managed_cluster_name = config.get("managedClusterName")
if managed_cluster_name is None:
    managed_cluster_name = prefix_name

# Create AKS cluster
# SEE: https://www.pulumi.com/docs/reference/pkg/azure-native/containerservice/managedcluster/
# TODO: https://www.pulumi.com/docs/reference/pkg/azure-native/containerservice/managedcluster/#createupdate-aad-managed-cluster-with-enableazurerbac
# TODO: https://www.pulumi.com/docs/reference/pkg/azure-native/network/azurefirewall/
# TODO: https://www.pulumi.com/docs/reference/pkg/azure-native/containerservice/managedcluster/#managedclusteraadprofile
managed_cluster = containerservice.ManagedCluster(
    managed_cluster_name,
    resource_group_name=resource_group_obj.name,
    addon_profiles={},
    agent_pool_profiles=[containerservice.ManagedClusterAgentPoolProfileArgs(
        enable_node_public_ip=False,
        name=system_pool_profile.get("name"),
        count=system_pool_profile.get("count"),
        max_pods=system_pool_profile.get("max_pods"),
        mode=system_pool_profile.get("mode"),
        node_labels=system_pool_profile.get("node_labels"),
        os_disk_size_gb=system_pool_profile.get("os_disk_size_gb"),
        os_type=system_pool_profile.get("os_type"),
        type=system_pool_profile.get("type"),
        vm_size=system_pool_profile.get("vm_size"),
        vnet_subnet_id=config.get("subnet_id"),
    )],
    api_server_access_profile=containerservice.ManagedClusterAPIServerAccessProfileArgs(
        enable_private_cluster=True,
    ),
    enable_rbac=True,
    kubernetes_version=config.get("kubernetes_version"),
    linux_profile={
        "admin_username": config.get("aks_admin_username"),
        "ssh": {
            "public_keys": [{
                "key_data": ssh_key.public_key_openssh,
            }],
        },
    },
    identity=containerservice.ManagedClusterIdentityArgs(
        type=containerservice.ResourceIdentityType.SYSTEM_ASSIGNED),
    dns_prefix=resource_group_obj.name,
    network_profile=containerservice.ContainerServiceNetworkProfileArgs(
        network_plugin="azure",
        pod_cidr=aks_network_profile.get("pod_cidr"),
        service_cidr=aks_network_profile.get("service_cidr"),
        docker_bridge_cidr=aks_network_profile.get("docker_bridge_cidr"),
        dns_service_ip=aks_network_profile.get("dns_service_ip"),
        outbound_type="userDefinedRouting",
    ),
    node_resource_group=f"MC_{managed_cluster_name}_westus",
    service_principal_profile={
        "client_id": config.get("clientId"),
        "secret": config.get("clientSecret")
    },
    tags={
        'CostTech': 'aks',
        **tags.standard_tags
    }
)

# SEE: https://www.pulumi.com/docs/reference/pkg/azure-native/containerservice/agentpool/#agentpoolmode
user_agent_pool = containerservice.AgentPool(
    "userPool",
    agent_pool_name=user_pool_profile.get("name"),
    count=user_pool_profile.get("count"),
    max_pods=user_pool_profile.get("max_pods"),
    mode=user_pool_profile.get("mode"),
    node_labels=user_pool_profile.get("node_labels"),
    enable_node_public_ip=False,
    # enable_encryption_at_host=True,
    os_type=user_pool_profile.get("os_type"),
    os_disk_size_gb=user_pool_profile.get("os_disk_size_gb"),
    os_disk_type=user_pool_profile.get("os_disk_type"),
    resource_group_name=prefix_name + "-rg",
    resource_name_="primerai-aks-dev-aks7f632331",
    type=user_pool_profile.get("type"),
    vm_size=user_pool_profile.get("vm_size"),
    vnet_subnet_id=config.get("subnet_id")
)

# SEE: https://www.pulumi.com/docs/reference/pkg/azure-native/storage/storageaccount/#storageaccountcreate
minio_account = storage.StorageAccount(
    "minio",
    account_name="aksminiorandomname",
    resource_group_name=f"MC_{managed_cluster_name}_westus",
    location=location,
    # kind="StorageV2",
    kind="Storage",
    minimum_tls_version="TLS1_2",
    sku=storage.SkuArgs(
        name="Standard_LRS",
    ),
    network_rule_set=storage.NetworkRuleSetArgs(
        bypass="AzureServices",
        default_action="Deny",
        ip_rules=[],
        virtual_network_rules=[storage.VirtualNetworkRuleArgs(
            virtual_network_resource_id=config.get("subnet_id"),
        )],
    ),
    tags={
        'CostTech': 'aks',
        **tags.standard_tags
    }
)

creds = pulumi.Output.all(resource_group_obj.name, managed_cluster.name).apply(
    lambda args:
    containerservice.list_managed_cluster_user_credentials(
        resource_group_name=args[0],
        resource_name=args[1])
)

output_private_ssh_key = pulumi.Output.all(ssh_key.private_key_pem)

# Export kubeconfig
encoded = creds.kubeconfigs[0].value
kubeconfig = encoded.apply(
    lambda enc: base64.b64decode(enc).decode())
pulumi.export("kubeconfig", kubeconfig)
pulumi.export("private_ssh_key", output_private_ssh_key)
EOF

See also

External links

Retrieved from "http://wiki.christophchamp.com/index.php?title=Pulumi&oldid=8109"