Data pipelines

Log and NetFlow ingest paths: UniFi gear and network devices to Splunk indexers, via HAProxy and Cribl Edge.

Two pipelines move data from the homelab edge to Splunk. The architecture is intentional: HAProxy load-balances toward Cribl Edge, which routes, transforms, and reduces volume before forwarding to Splunk over HEC.

The goal: 30–50% ingest reduction without losing security signal.

Log pipeline

UniFi network gear and application logs land in Splunk via Cribl Edge. HAProxy fronts the Cribl Edge cluster for high availability.

Coral dashed edges carry the data; the solid green edge is the physical syslog hop. Cribl Edge drops verbose fields, routes by event_type, enriches and masks — the indexer takes a smaller, cleaner payload.

NetFlow pipeline

NetFlow v9 / IPFIX from network devices follows the same shape on a different port. UDP is loss-tolerant by design, so HAProxy distributes rather than fails over.

Cribl pipelines de-duplicate, parse flow records, and aggregate by tuple before forwarding.

What lives where

Layer	Provisioned by	Configured by	Source repo
Proxmox host / VMs / LXCs	terraform-proxmox	ansible-proxmox	both
HAProxy	(Ansible role)	ansible-proxmox-apps	apps repo
Cribl Edge	(Ansible role)	ansible-proxmox-apps	apps repo
Splunk Enterprise	(manual / Ansible)	ansible-splunk	splunk repo
Cribl pipelines	(manual / Cribl pack)	cc-edge-* packs	pack repos
Splunk knowledge objects	n/a	Splunk TA (AI observability)	TA repo

DR posture

Splunk Cloud failover is provisioned via terraform-aws — AWS resources (EC2, S3, Route 53) that come up cold and accept the same HEC traffic if the home cluster is offline. Cribl Edge routes can be flipped to point at the AWS endpoint with a single config change.