Archived
Drop ZFS-replication as an interim HA step; jump straight to Ceph
Simplifies the storage model to one long-term tier (Ceph) instead of two. Node 1 runs local ZFS only until nodes 2/3 join, at which point Ceph goes live and VMs migrate onto it directly.
This commit is contained in:
1 parent
a854412117
commit
24a0047fa8
4 files changed
+70
-58
No files matched your search
+18
-19
@@ -14,19 +14,16 @@ new hardware. Before reusing that workaround on the new box:
|
||||
|
||||
## End goal
|
||||
|
||||
3-node Proxmox VE cluster with HA-managed VMs. Two ways to get VM disks
|
||||
available on more than one node (see `02-storage-zfs-ceph.md`):
|
||||
3-node Proxmox VE cluster with HA-managed VMs backed by **Ceph** — true
|
||||
distributed shared storage, sync replication, near-zero RPO on failover
|
||||
(see `02-storage-zfs-ceph.md`). Ceph needs 3+ nodes and a fast dedicated
|
||||
network, so it can't exist until nodes 2 and 3 are up.
|
||||
|
||||
1. **Ceph** — true distributed shared storage, sync replication, needs 3+
|
||||
nodes and a fast dedicated network. Zero/near-zero RPO on failover.
|
||||
2. **ZFS + storage replication** — local ZFS pool per node, Proxmox
|
||||
replicates VM disks between nodes on a schedule (as often as every
|
||||
minute). Lighter weight, async — failover loses whatever changed since
|
||||
last replication.
|
||||
|
||||
Starting point: ZFS + replication (lighter, works from node 1 onward).
|
||||
Ceph is the upgrade path once 3 nodes exist and/or zero-RPO failover
|
||||
matters enough to justify the overhead.
|
||||
Deliberately no intermediate "ZFS + storage replication" HA step. Node 1
|
||||
runs local ZFS for boot + VM storage with no cluster-wide HA until Ceph
|
||||
goes live — as soon as nodes 2/3 join, VMs move onto Ceph rather than
|
||||
adopting ZFS replication as a stopgap. Simpler end state, one storage
|
||||
model to operate instead of two.
|
||||
|
||||
## Cluster fundamentals (apply from node 1 onward)
|
||||
|
||||
@@ -44,14 +41,16 @@ matters enough to justify the overhead.
|
||||
## Rollout sequence
|
||||
|
||||
1. Build node 1 per `01-hardware-node1.md` — fresh PVE install on ZFS boot
|
||||
mirror, local ZFS "tier 2" pool for VM disks, Ceph-earmarked disks left
|
||||
idle or as a temporary ZFS pool.
|
||||
mirror, Ceph-earmarked disks left idle (or as a temporary local ZFS
|
||||
pool, to be wiped later — see `02-storage-zfs-ceph.md`).
|
||||
2. Migrate VMs off old hardware via `vzdump` → copy backups → `qmrestore`
|
||||
onto the new ZFS storage (converts disks to ZVOLs).
|
||||
onto the new ZFS storage (converts disks to ZVOLs). No HA yet — single
|
||||
node.
|
||||
3. Apply hardening (`04-security-hardening.md`) and networking
|
||||
(`03-networking.md`) before exposing the node beyond the LAN.
|
||||
4. Add nodes 2 and 3 identically (same disk/network layout).
|
||||
5. Join cluster, stand up dedicated corosync network, configure
|
||||
replication and HA groups.
|
||||
6. Optionally migrate the Ceph-earmarked disks from temporary ZFS to real
|
||||
Ceph OSDs once 3 nodes are up.
|
||||
5. Join cluster, stand up dedicated corosync network.
|
||||
6. Wipe the Ceph-earmarked disks (if used as temporary ZFS) and
|
||||
initialize Ceph across all 3 nodes.
|
||||
7. Migrate VMs from local ZFS onto Ceph-backed storage, then configure HA
|
||||
groups.
|
||||
Reference in new issue
Block a user