DevOps VM automation enabled by vSphere with Tanzu VM Service

With the launch of vSphere 7 Update 2a, VMware vSphere with Tanzu delivers the VM Service to provide DevOps teams with the capability to provision and manage virtual machines using the embedded Kubernetes API.  Prior to this release the VM Service was used exclusively to manage the VMs that make up Tanzu Kubernetes Grid (TKG) clusters.   Now Developers can orchestrate their own stand-alone VMs using open source and proprietary tools from the Kubernetes ecosystem.  IT Administrators provide governence of access, resource consumption and base image availability through the Supervisor Kubernetes cluster Namespace.  Modern applications may run through DevOps deployed TKG Clusters, Kubernetes pods natively on vSphere and now standalone Virtual Machines - all orchestrated through Kubernetes.   

For those that want to skip the blog and see it in action, the link to the video is Here and embedded at the end of the blog.

A bit about Kubernetes Custom Resources.

The Kubernetes API is extensible and allows developers to add their own Kind of object that can hold configuration and use standard operations like Create, Delete, Get, Describe, etc.  Custom Resources on their own don't really do anything other than hold metadata.   Developers will also create Controllers that watch specific custom resources and take action based on changes in the resource.  vSphere with Tanzu makes extensive use of Custom Resources and Controllers (also collectively known as Operators).   In this blog/video I previously described their use as part of our ClusterAPI implementation in the Tanzu Kubernetes Grid Service - Here.   The VM Service implements a VirtualMachine and a VirtualMachineService Custom Resource along with a set of Controllers to lifecycle manage VMs.   The VirtualMachineService Custom Resource holds the configuration for providing Ingress and Load Balancing into VMs instantiated through the VirtualMachine Custom Resource.

IT Administrative Governance

vSphere with Tanzu introduced the concept of a vSphere Namespace.  This construct spans vCenter and the Supervisor Kubernetes Cluster to provide an envelope for Admins to attach policy to a user "Sandbox".   Previously providing access controls, storage and network policy,  as well as resource limits - the Namespace now allows for the assignment of Virtual Machine classes and Content Libraries containing accessible VM base images.   VM Classes are part of the VM specification developers would submit to instantiate the VMs and Admins can put guardrails on VM sizing by assigning specific Classes to the Developer's Namespace.  Similarly, Admins may wish to restrict the use of VM base images to specific approved images.  Note: In the first release of VM Service VMware will provide a set of base images that have been tested and are known to work with the service.  Bring Your Own Image is the plan and we expect to make that available in the near future.

The VM Service includes a set of default VM Classes that IT Admins assign to the Developer's Namespace through the vSphere Client.   Best effort classes would allow overcommit, while guaranteed classes would set reservations enforced through Resource Pools and DRS.   So if an organization did not want to allow reserved resources the IT Admin would simply not assign any guaranteed classes to the Developer Namespace.

To make base images available to the Namespace, create a content library that includes the appropriate images.  For the first release, the VMware tested images are available in the VMware Cloud Marketplace  https://marketplace.cloud.vmware.com/.   Search for "VM Service".   Choose your image and subscribe.   Subscribing will give you a URL that can be used as part of the Subscribed Content Library configuration in the vSphere Client.  Images will be automatically updated in your CL if you desire.  

Once the Namespace is configured with the appropriate VM Classes and base image Content Library, your DevOps team is ready to create VMs.

The DevOps Experience

There are two components to the creation of the VM; the VirtualMachine custom resource and the ConfigMap.

VirtualMachine

apiVersion: vmoperator.vmware.com/v1alpha1
kind: VirtualMachine
metadata:
  name: centos-cloudinit-example
  namespace: vmservice
  labels:
    app: vmservice-centos-vm
spec:
  networkInterfaces:
  - networkName:  ""
    networkType:  nsx-t
  className: best-effort-small
  imageName: centos-stream-8-vmservice-v1alpha1.20210222
  powerState: poweredOn
  storageClass: kubernetes-policy
  vmMetadata:
    configMapName: centos-cloudinit-test
    transport: OvfEnv

The VirtualMachine specification defines the networking to be used.  In this case we are using NSX, however users may deploy with the vDS based configuration and an external Load Balancer.  A blog/video on that setup is Here.   Additionally, the specification includes the base image, the storage class and the VMclass to be used.  Only storage classes, VMclasses and images assigned to the Namespace will be allowed.   This specification will get you a VM, but if you want to configure it beyond what is in the base image, you need to create a ConfigMap to hold that configuration and include it in the VM specification.   Virtual Machine configuration is defined in the ConfigMap and passed to the VM using OVF extraconfig.  Cloudinit will apply that configuration to the VM on its first boot.

ConfigMap

---
apiVersion: v1
kind: ConfigMap
metadata:
    name: centos-cloudinit-test
    namespace: vmservice
data:
  user-data: |
          I2Nsb3VkLWNvbmZpZwpjaHBhc3N3ZDoKICAgIGxpc3Q6IHwKICAgICAgbXdlc3Q6Vk13YXJlMSEKICAgIGV4cGlyZTogZmFsc2UKZ3JvdXBzOgogIC0gZG9ja2VyCnVzZXJzOgogIC0gZGVmYXVsdAogIC0gbmFtZTogbXdlc3QKICAgIHNzaC1hdXRob3JpemVkLWtleXM6CiAgICAgIC0gc3NoLXJzYSBBQUFBQjNOemFDMXljMkVBQUFBREFRQUJBQUFCQVFDbVBQbHZQa3YzK1VMUVZOQ3R2N0IrdXVuZkR5VHd0RnluWGdQMGl3emk4dzlTQTdtaTFiRnNZZFphbzd0NGVZRWVNMFY1dDRabXVtdmdHS29QSzFMKzVaNGxEWWoxbU9KUjdIUnN4TW9GYU1ibU1waVdJY2prWFhDTko0bUtqY1N5ZE9ITUhKTXRLRklqc1Y5VUN3KzhMZnpjQ2h3R0o5RWUxRXlLc1BQcEE0MlFVWjZoc205bmt0dU5KQy9ML1V4WURqUllxY0F3U3R3QjRxb0F3L2NzVmhQY0oweEFhVi9yZFNxMGpQeGRVQWFCYzVISjFMOHpYNm1GMWFINzRsSXkvZHRWRVQ1YnhpbExMTjNoM3ZrK3dSUDNxSWYrN09vWE5TeXBaVldHNmFNK3MzeUYwK016Ti9XU09YZ0hSZm9GSGhFd0RnaC9OWE1oSXNad0V0eEYgaW1wb3J0ZWQtb3BlbnNzaC1rZXkKICAgIHN1ZG86IEFMTD0oQUxMKSBOT1BBU1NXRDpBTEwKICAgIGdyb3Vwczogc3VkbywgZG9ja2VyCiAgICBzaGVsbDogL2Jpbi9iYXNoCm5ldHdvcms6CiAgdmVyc2lvbjogMgogIGV0aGVybmV0czoKICAgICAgZW5zMTkyOgogICAgICAgICAgZGhjcDQ6IHRydWUK
  hostname: centos-cloudinit-example

The ConfigMap doesn't show much.  You see some metadata and a base64 encoded user-data string.  This string encapsulates the configuration used by cloudinit.  Converted from the base64 string to readable text shows the configuration provided to the VM.

User-Data

#cloud-config
chpasswd:
    list: |
      mwest:VMware1!
    expire: false
groups:
  - docker
users:
  - default
  - name: mwest
    ssh-authorized-keys:
      - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCmPPlvPkv3+ULQVNCtv7B+uunfDyTwtFynXgP0iwzi8w9SA7mi1bFsYdZao7t4eYEeM0V5t4ZmumvgGKoPK1L+5Z4lDYj1mOJR7HRsxMoFaMbmMpiWIcjkXXCNJ4mKjcSydOHMHJMtKFIjsV9UCw+8LfzcChwGJ9Ee1EyKsPPpA42QUZ6hsm9nktuNJC/L/UxYDjRYqcAwStwB4qoAw/csVhPcJ0xAaV/rdSq0jPxdUAaBc5HJ1L8zX6mF1aH74lIy/dtVET5bxilLLN3h3vk+wRP3qIf+7OoXNSypZVWG6aM+s3yF0+MzN/WSOXgHRfoFHhEwDgh/NXMhIsZwEtxF imported-openssh-key
    sudo: ALL=(ALL) NOPASSWD:ALL
    groups: sudo, docker
    shell: /bin/bash
network:
  version: 2
  ethernets:
      ens192:
          dhcp4: true

We are simply setting up a user and groups, while injecting an authorized ssh key into the VM.   That's it.   

For more information and to see how this works, check out the VM Service technical overview video:  

https://www.youtube.com/watch?v=6Fz6rY_wzwQ

For an even deeper dive into the developer experience with the vSphere with Tanzu VM Service, check out this fantastic blog and video:  https://core.vmware.com/blog/introducing-virtual-machine-provisioning-kubernetes-vm-service