cloudinit does not run for qemu/kvm systems created by terraform and libvirt provider
2
votes
2
answers
1455
views
I'm trying to provision a VM on qemu/kvm hypervisor using cloudinit with terraform and the libvirt provider. I can get the machine to start, but the cloudinit is not getting kicked off. I know that the user-data being used will work as I've tested it without using terraform, but instead using kvm to spin up a machine with a second terminal running a web server to server up the user-data file. All of this is being performed on Ubuntu 18.04.
I've tried this using cloud images for ubuntu and centos. Both will create the VM and boot up to the login prompt. Neither will actually provision the contents of the user-data though. I've also tried this using a lower version of the libvirt provider (of those still available) to the same results.
I've done some extensive searching trying to see if anyone else has similar issues. Most of the sites/questions I've found were from StackExchange sites, as well as github issues and bug reports, which I didn't capture to document unfortunately. All of them tend to be something in the user-data that is missing. None of them have been using the version of the libvirt provider I am, but instead almost always version 0.6.2. I tried to use that version of the provider in my main.tf file, but the init command returns an error that the version doesn't exist and can't be downloaded.
main.tf (only difference between centos and ubuntu is the cloud image file/location and prefix variable)
terraform {
required_providers {
libvirt = {
source = "dmacvicar/libvirt"
version = "0.7.0"
}
}
}
# instantiate the provider
provider "libvirt" {
uri = "qemu:///system"
}
variable "prefix" {
default = "terraform_centos"
}
data "template_file" "user_data" {
template = file("${path.module}/cloud_init.cfg")
}
resource "libvirt_cloudinit_disk" "commoninit" {
name = "commoninit.iso"
user_data = data.template_file.user_data.rendered
}
resource "libvirt_volume" "qcow_volume" {
name = "${var.prefix}.img"
source = "https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2 "
format = "qcow2"
}
resource "libvirt_domain" "centos" {
name = var.prefix
vcpu = 2
memory = 4096
disk {
volume_id = libvirt_volume.qcow_volume.id
}
cloudinit = libvirt_cloudinit_disk.commoninit.id
console {
type = "pty"
target_port = "0"
target_type = "serial"
}
console {
type = "pty"
target_type = "virtio"
target_port = "1"
}
network_interface {
network_name = "default"
}
}
user-data file (as with main.tf, only changes between the two distros being tested is the hostname)
-yaml
#cloud-config
autoinstall:
version: 1
identity:
hostname: terraform_centos
username: vagrant
password: $6$dnWt7N17fTD$8.m3Rgf400iSyxLa/kUtunGUgE3N4foSg/y31HNnsGBUTpoMOmS3O9U/nJFvZjXpQTrLFrAcK5vok5EI0KZA90
locale: en_US
keyboard:
layout: us
ssh:
install-server: true
authorized-keys:
- ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzIw+niNltGEFHzD8+v1I2YJ6oXevct1YeS0o9HZyN1Q9qgCgzUFtdOKLv6IedplqoPkcmF0aYet2PkEDo3MlTBckFXPITAMzF8dJSIFo9D8HfdOV0IAdx4O7PtixWKn5y2hMNG0zQPyUecp4pzC6kivAIhyfHilFR61RGL+GPXQ2MWZWFYbAGjyiYJnAmCP3NOTd0jMZEnDkbUvxhMmBYSdETk1rRgm+R4LOzFUGaHqHDLKLX+FIPKcF96hrucXzcWyLbIbEgE98OHlnVYCzRdK8jlqm8tehUc9c9WhQ== vagrant insecure public key
allow-pw: true
storage:
layout:
name: direct
packages:
- gcc
- build-essential
late-commands:
- "echo 'vagrant ALL=(ALL) NOPASSWD: ALL' >> /target/etc/sudoers.d/vagrant"
- "chmod 440 /target/etc/sudoers.d/vagrant"
Finally, the kvm command that actually gets cloudinit to work. This is after starting a quick python web server in another shell. Along with mounting the iso image into /mnt so that the kernel/initrd will be accessible.
-bash
kvm -no-reboot -m 4096 -drive file=focal.img,format=raw,cache=none,if=virtio -cdrom ~/isoImages/ubuntu-20.04.5-live-server-amd64.iso -kernel /mnt/casper/vmlinuz -initrd /mnt/casper/initrd -append 'autoinstall ds=nocloud-net;s=http://_gateway:3003/ ' -vnc :1
Qemu/kvm version info
$ virsh version
Compiled against library: libvirt 4.0.0
Using library: libvirt 4.0.0
Using API: QEMU 4.0.0
Running hypervisor: QEMU 2.11.1
Terraform version info
$ terraform version
Terraform v1.3.6
on linux_amd64
+ provider registry.terraform.io/dmacvicar/libvirt v0.7.0
+ provider registry.terraform.io/hashicorp/template v2.2.0
Because I'm using cloud images for the two distros, there isn't a default username/password that I can log in with to check logs on the VM directly. I do have access to the console via virt-manager and the virsh console command. Both are sitting at a login prompt, at which the vagrant user returns a login incorrect message.
If any further information is needed, please let me know. I'm open to suggestions on what needs to be done to get this to work.
Asked by Jonathan Heady
(121 rep)
Dec 16, 2022, 07:27 PM
Last activity: Nov 7, 2023, 11:47 PM
Last activity: Nov 7, 2023, 11:47 PM