Load Balancers¶

Load balancing in the cluster is handled entirely by Cilium's LoadBalancer IP Address Management (LBIPAM). There is no MetalLB -- Cilium's L2 announcements provide the same functionality with fewer components and tighter integration with the CNI.

How Cilium LBIPAM Works¶

When a Kubernetes Service of type LoadBalancer is created, Cilium assigns it an IP address from the configured pool and responds to ARP requests for that IP on all Linux nodes. This allows clients on the LAN to reach the service at the assigned IP.

flowchart TB
    subgraph Pool["CiliumLoadBalancerIPPool<br/>192.168.0.220 - 192.168.0.239"]
        IP1[.220 - CoreDNS]
        IP2[.221 - nginx-internal]
        IP3[.222 - LLDAP]
        IP4[.226 - Mosquitto]
        IP5[.227 - Home Assistant]
        IP6[.228 - Matter Server]
        IP7[.229 - Jellyfin]
        IP8[.230 - OTBR]
        IP9[.231 - nginx-external]
        IP10[.238 - envoy-internal]
        IP11[.239 - envoy-external]
    end

    subgraph Cilium["Cilium Agent (per node)"]
        L2[L2 Announcement<br/>ARP Responder]
    end

    Client((LAN Client)) -->|"ARP: Who has<br/>192.168.0.238?"| L2
    L2 -->|"ARP Reply:<br/>Node MAC"| Client
    Client -->|"IP Traffic"| IP10

    classDef pool fill:#00b894,stroke:#00a381,color:#fff
    class IP1,IP2,IP3,IP4,IP5,IP6,IP7,IP8,IP9,IP10,IP11 pool

IP Assignment Methods¶

There are two ways a Service gets an IP from the pool:

Automatic assignment: If no specific IP is requested, Cilium picks the next available IP from the pool.

Explicit assignment: Services can request a specific IP using the lbipam.cilium.io/ips annotation:

apiVersion: v1
kind: Service
metadata:
  annotations:
    lbipam.cilium.io/ips: "192.168.0.238"
spec:
  type: LoadBalancer

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: envoy-internal
spec:
  infrastructure:
    annotations:
      lbipam.cilium.io/ips: "192.168.0.238"

IP Pool Configuration¶

The LBIPAM pool is defined by two Cilium CRDs:

apiVersion: cilium.io/v2alpha1
kind: CiliumL2AnnouncementPolicy
metadata:
  name: policy
spec:
  loadBalancerIPs: true
  nodeSelector:
    matchLabels:
      kubernetes.io/os: linux
---
apiVersion: cilium.io/v2alpha1
kind: CiliumLoadBalancerIPPool
metadata:
  name: pool
spec:
  allowFirstLastIPs: "Yes"
  blocks:
    - start: 192.168.0.220
      stop: 192.168.0.239

IP Allocation Table¶

Current LoadBalancer IP assignments across the cluster:

Why Certain Services Need LoadBalancer IPs¶

Not all services need a dedicated LoadBalancer IP. Most applications use HTTPRoute/Ingress and are accessible through the Envoy gateways. However, some services require their own IP because:

Adding a New LoadBalancer Service¶

To assign a specific IP to a new LoadBalancer service:

For a Standard Service¶

apiVersion: v1
kind: Service
metadata:
  name: my-service
  annotations:
    lbipam.cilium.io/ips: "192.168.0.223"  # Pick an available IP
spec:
  type: LoadBalancer
  ports:
    - port: 8080
      targetPort: 8080

For an Envoy Gateway¶

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: my-gateway
spec:
  gatewayClassName: envoy
  infrastructure:
    annotations:
      lbipam.cilium.io/ips: "192.168.0.223"
  listeners:
    - name: https
      protocol: HTTPS
      port: 443

For an App-Template (bjw-s) Service¶

service:
  app:
    controller: my-app
    type: LoadBalancer
    annotations:
      lbipam.cilium.io/ips: "192.168.0.223"
    ports:
      http:
        port: 8080

Troubleshooting¶

Check IP Pool Status¶

# View the IP pool and available IPs
kubectl get ciliumloadbalancerippools

# Check detailed pool status
kubectl describe ciliumloadbalancerippool pool

List All LoadBalancer Services¶

# List all LoadBalancer services with their external IPs
kubectl get svc -A --field-selector spec.type=LoadBalancer

Check L2 Announcement Status¶

# Verify the L2 announcement policy
kubectl get ciliuml2announcementpolicies

# Check which node is announcing a specific IP
kubectl get leases -n kube-system | grep cilium-l2

IP Conflict Troubleshooting¶

# Find services stuck in Pending
kubectl get svc -A --field-selector spec.type=LoadBalancer | grep Pending

# Check events for IP allocation issues
kubectl events -A --field-selector reason=IPAllocationFailed

Verify ARP Responses¶

# From a LAN machine, check ARP for a LoadBalancer IP
arp -n 192.168.0.238

# Or use arping to test
arping -c 3 192.168.0.238