Over the weekend, I created a home lab. I had an aging PC that had windows on it and I decided to wipe it away and install Ubuntu.
I created a home lab late last year on my main PC, but I didn't like having all of that infrastructure on a PC where I did general purpose programming and gaming. Right off the bat, Windows hypervisor and the VM's I'd need for a proper lab took up precious RAM. And there was also the matter of having the VM's up and running as nodes in my Kubernetes cluster, having Rancher running etc. It all seemed like extra baggage for my PC which I wanted to be as lean as possible and ready to handle large tasks (like running games) using all of the resources in the machine.
So this PC has 16GB of RAM and an i5 4-core processor. It's unfortunate that I have 32GB of DIMMs installed on the box. My mother board says it supports 32GB of RAM but for some reason I can't get the BIOS to recognize it. Turns out to a CLI-only Ubuntu, this is all of the power I need. Anyway, figuring out how to use the other 16GB of ram is an Area of Research! The box has an old SSD I pulled from my old Apple Mac Pro and performance of everything is just lightning.
When it came to a hypervisor to use, I chose kvm. It seemed like the right choice for my needs and did't take a lot of know-how to get working. This link helped: https://help.ubuntu.com/community/KVM/Installation
When it came to VM's to create, I used this Minimal CD image of Ubuntu: https://help.ubuntu.com/community/Installation/MinimalCD. During the installation of each VM, I chose OpenSSH server only since that's all I needed. VM installation was still manual and a little time-consuming. At some point I hope to automate it, but since I only have 16GB of RAM, I'm only creating 203 VM's as Kubernetes worker nodes and that will be that.
A few useful installs:
- sudo apt install libguestfs-tools
- sudo apt install virtinst
- sudo apt install libosinfo-bin
- sudo apt install cpu-checker
I decided to use Rancher as the k8s cluster management layer since we use that at work and I figure it would be a good idea to get more familiar with it. Obviously the new server needed docker and this article was helpful in describing how to get docker to install properly on Ubuntu: https://docs.docker.com/engine/install/ubuntu/. It turns out just saying "apt install docker.io" is not the best choice. You need a few extra steps.
Rancher was up and running without much effort. Rancher runs in a container directly on the host, on a server I call SINGULARITY. I have two VM's called star01 and star02 which run as work nodes. star01 also runs etcd and the control plane. I'll probably create a new VM star03 soon so I can experiment with deploying various software in clusters which require 3+ nodes, but I can prob get by with 2-nodes for a while.
An open item is having star01/02 DNS available on my LAN without using a bridged network. I tried implementing the technique described here, but it didn't work on my first try. I'll have to try again. It would probably help if I understood NetworkManager and DNS better. More areas of research!
A few things I'm interested in learning about in the short term:
- How to create a docker image that manipulates mounts on the host for external storage for configuration. Rancher does this and it's kind of magic to me right now.
- Figure out how to have all of my kvm guest hostnames recognized on the host, if not my entire home LAN.
- How to backup my Rancher and cluster configuration so I can restore it if I want to blow away my lab and start from scratch. This blog post is good documentation if I want to start over, but obviously I'd rather automate away the toil.
Some useful commands:
- Make sure your kvm guests restart after host reboot: virsh autostart vmName
- List kvm guests and their IP addresses: virsh net-dhcp-leases default
- Dump network details: sudo virsh net-dumpxml default
- Edit network details: sudo virsh net-edit default
Ciao for now.