← All work
active · 2 min read

NetBox Ingest

Production Go service that correlates infrastructure from Proxmox, DNS, and UniFi into unified NetBox entries: multi-source deduplication, continuous sync, and Prometheus metrics for full visibility.

GoKubernetesNetBoxObservability
Status
active
Active since
Aug 2025
Last updated
July 2026
Changes shipped
15

The source of truth for infrastructure inventory; continuously syncing and refined.

Infrastructure inventory rots the moment it’s maintained by hand. NetBox Ingest keeps it honest by pulling from the systems that actually know the truth (the hypervisor, DNS, and the network controller) and reconciling them into a single source of record.

How it works

NetBox Ingest data flow Three independent collectors (Proxmox, DNS, and UniFi) feed a reconcile stage that de-duplicates and correlates records into one canonical entry in NetBox. A Prometheus tap exposes per-source contribution and conflict metrics. Sources Reconcile System of record Proxmox VMs & nodes DNS host records UniFi network clients Dedup + correlate one canonical entry NetBox canonical inventory Prometheus metrics

Each source is collected independently behind a common interface, then handed to a correlation engine that matches records describing the same real thing across sources and consolidates them into one canonical entry. The reconciled result is written to NetBox; a Prometheus tap on the sync stage exposes how much each source contributed and where they disagreed. The whole pass is a single Go service, deployed to the cluster and run continuously.

Key decisions

  • Reconcile, don’t overwrite: a dedicated correlation engine merges what multiple sources say about one device rather than letting the last writer win. The cost is matching logic per resource type; the payoff is one trustworthy entry instead of three conflicting ones.
  • Idempotent sync: a run computes desired state and converges to it, so re-running changes nothing when nothing changed. That makes the service safe to run on a tight schedule and keeps NetBox’s own change log free of churn.
  • Fail safe on an empty source: an offboard zero-guard refuses to mark inventory offline or prune it when a source returns nothing, because a flaky API looks identical to “everything is gone.” A blip can’t wipe the inventory.
  • Observability is part of the product: exposing per-source contribution and conflict as metrics means you can see not just what exists, but how confident the system is about it, and spot a degraded source before it corrupts the record.

Recent changes

  • Reconcile existing vms devices
  • Ansible managed tag authority
  • Make netbox sync idempotent
  • Unifi client offboarding
  • Primary ip linking
  • Fix vm ip interface assignment