Hardware Inventory

The cluster is built from a mix of x86 and ARM hardware, mounted in an open-frame 12U rack together with a patch panel, a PDU, and a UPS. AMD mini PCs run the control plane and Ceph, a Dell R630 provides bulk compute and ZFS storage, a GPU workstation runs AI workloads, and Intel nodes and Raspberry Pis fill out the worker pool.

Server rack

All nodes are on VLAN 20 with static DHCP reservations (terraform/unifi/reservations.tf). CPU and memory figures below are what Kubernetes reports.

Compute Nodes

Control Plane + Workers

These nodes run the control plane and schedule workloads: the control-plane taint is removed in talos/pitower/control-plane/01-cluster.yaml, with systemReserved/kubeReserved set so app load cannot starve the API server. The VIP 10.20.10.0 floats across all three (https://10.20.10.0:6443).

HostnameHardwareIPCPURAMStorageSchematic
worker-01AMD Ryzen mini PC10.20.10.116 threads16 GBNVMe boot + SATA SSD (Ceph OSD)amd
worker-02AMD Ryzen mini PC10.20.10.216 threads32 GBNVMe boot + SATA SSD (Ceph OSD)amd
worker-03AMD Ryzen mini PC10.20.10.316 threads32 GBNVMe boot + SATA SSD (Ceph OSD)amd

Intel Workers

HostnameIPCPURAMBoot diskNotesSchematic
worker-0410.20.10.44 cores8 GBeMMCdedicated=media-home taintintel
worker-0510.20.10.54 cores8 GBSamsung SSDAlso on the untagged LAN (networking/lan); no VT-xintel
worker-0610.20.10.64 cores8 GBSamsung SSDAlso on the untagged LAN (networking/lan); no VT-xintel

worker-07 (Dell R630)

A bare-metal Dell R630 (ex proxmox-01, see the migration record) at 10.20.10.7, on a 10G Intel X710 port.

PropertyValue
CPU48 threads
RAM~755 GiB usable (ZFS ARC capped at 64 GiB)
DisksSix 745 GiB SAS SSDs and four ~600 GB 10K SAS HDDs behind the PERC in HBA mode
Talos system diskmd RAID1 across two SSDs (RAIDArrayConfig, selected by WWID)
ZFS fastFour SSDs as two mirrors: /var/mnt/extra, /var/mnt/runners, Garage metadata
ZFS hddFour HDDs as raidz1: /var/mnt/media, Garage blocks
Schematicr630 (adds the zfs extension)

The OpenEBS classes openebs-hostpath-fast, -runners, and -media only provision here, and the self-hosted home-ops CI runners are pinned to it. It also runs Garage S3. ZFS is administered from a privileged hostPID pod with nsenter, since Talos has no shell; system/zfs-scrub scrubs both pools monthly.

worker-ai-01 (GPU)

A bare-metal ASUS ProArt B850 board with an NVIDIA RTX 3090 Ti at 10.20.10.11, dual-booted with Bazzite.

PropertyValue
CPUAMD, 16 threads
RAM64 GB
System disk1 TB Kingston NV3 (shared with Bazzite; Talos EPHEMERAL capped at 200 GiB)
Models disk2 TB Samsung 9100 Pro, Talos user volume models at /var/mnt/models (openebs-hostpath-models)
Schematicnvidia (open GPU kernel modules + container toolkit)
Taintdedicated=gpu:NoSchedule

The GPU is shared through the NVIDIA DRA driver (system/dra-driver-nvidia-gpu) and can be passed through to a KubeVirt VM with vfio-pci. system/nvidia-power-limit caps it at 300 W. Because it is often booted into Bazzite, it is excluded from Cilium's L2 and BGP announcements.

Raspberry Pi Workers

HostnameIPModelRAMBoot diskSchematic
worker-0810.20.10.8Raspberry Pi 4 (PoE HAT)4 GB128 GB USB SSDrpi-poe
worker-0910.20.10.9Raspberry Pi 4 (PoE HAT)4 GB128 GB USB SSDrpi-poe
worker-1010.20.10.10Raspberry Pi 4 (PoE HAT)4 GB128 GB USB SSDrpi-poe

The rpi-poe schematic uses the sbc-raspberrypi overlay and sets PoE HAT fan thresholds. KubeVirt workloads are kept off these arm64 nodes.

Network Equipment

DevicePurpose
UniFi Cloud Gateway FiberRouter, VLANs, DHCP, BGP peer (ASN 64512) for LoadBalancer and pod routes
TP-Link PoE switchPowers the Raspberry Pi nodes
UniFi access pointsWi-Fi
Patch panelFront of the rack

Networks (terraform/unifi/networks.tf):

NetworkVLANSubnet
Defaultuntagged192.168.0.0/24
home1010.10.0.0/16
servers2010.20.0.0/16
management5010.50.0.0/24
iot10110.101.0.0/16

Storage

DevicePurpose
Ceph OSDsOne SATA SSD (~500 GB) on each of worker-01..03, ceph-block default StorageClass
worker-07 ZFSfast and hdd pools, OpenEBS hostpath classes and Garage S3
worker-ai-01 NVMe2 TB models volume
Synology NAS (data, 10.20.10.100)NFS for media and bulk data

Power

DevicePurpose
PowerWalker UPSIn the rack
PDUPower distribution in the rack
Eaton 5S, CyberPower PR1500LCDRT2UUPSes served by the NUT host (nut, Ansible role nut) and scraped by monitoring/nut-exporter

Network Topology

graph LR
    Internet -->|WAN| GW[UniFi Cloud Gateway Fiber]
    GW -->|VLAN 20 + LAN| SW[Switching]

    SW --> CP[worker-01..03<br/>AMD]
    SW --> IN[worker-04..06<br/>Intel]
    SW --> R630[worker-07<br/>R630, 10G]
    SW -->|PoE| PI[worker-08..10<br/>Pi 4]
    SW --> AI[worker-ai-01<br/>GPU]
    SW --> NAS[Synology NAS]

    CP & IN & R630 & PI -.->|BGP| GW