Docs rewrite

This commit is contained in:
Francesco Zimbolo
2026-04-24 12:58:14 +00:00
parent 9e294aea1a
commit 8477f5b592
7 changed files with 425 additions and 134 deletions

View File

@@ -1,21 +1,84 @@
# 05 - Linstor & Storage Implementation
This document details the configuration enforcement executed by the Linstor roles.
# 05 - Linstor & Storage Deep Dive
## LVM Global Filter
To prevent PVE from detecting double LVM signatures (raw block vs DRBD layer), the playbook injects the following global filter into `/etc/lvm/lvm.conf`:
`global_filter =[ "r|^/dev/drbd|", "r|^/dev/mapper/[lL]instor|" ]`
## Storage Provisioning
The `community.general.lvg` task targets raw block devices strictly when the target string starts with `/dev/`. **The playbook does not invoke `wipefs`.** If residual partition tables exist on the target block device, the LVM module will purposefully fail to prevent data destruction.
To prevent PVE from scanning DRBD and Linstor mapper devices as additional LVM physical volumes (which causes duplicate signature warnings and can confuse `vgscan`), the playbook injects a global filter into `/etc/lvm/lvm.conf`:
## Double Provisioning Protection (local-lvm)
By default, Proxmox creates a local storage entry named `local-lvm` pointing to the `pve/data` thin-pool. If the playbook detects that you are reusing `pve/data` as a Linstor backend, it automatically executes `pvesm remove local-lvm`. This prevents users from accidentally deploying local VMs to the underlying pool, which causes silent thin-pool exhaustion and DRBD corruption. The `local` directory storage (`/var/lib/vz`) is intentionally retained for ISOs and backups.
```
global_filter = [ "r|^/dev/drbd|", "r|^/dev/mapper/[lL]instor|" ]
```
## HA Controller Topology
Linstor Controller HA is handled via `drbd-reactor`.
1. A 200M DRBD resource (`linstor_db`) is spawned across all nodes.
2. The default `linstor-controller` systemd service is disabled.
3. A `var-lib-linstor.mount` unit is deployed.
4. The local `/var/lib/linstor` mount point is secured with the immutable flag (`chattr +i`). This guarantees that if the DRBD resource fails to mount, the local filesystem rejects writes, preventing database split-brains.
5. `drbd-reactor` manages the floating VIP and promoter plugin logic.
This is applied on all nodes before any VG or pool creation.
## Disk Wipe (Optional)
When `wipe_linstor_disks: true`, the playbook runs the following against every raw block device target before creating LVM structures:
```bash
wipefs -a /dev/sdX # removes filesystem and RAID signatures
sgdisk --zap-all /dev/sdX # destroys GPT and MBR partition tables
```
This only affects targets defined as `/dev/...` paths. Existing thin pool targets (e.g. `pve/data`) are never touched. **Set this flag only on first deployment — never on a running cluster.**
## Storage Pool Provisioning
The `linstor_storage_pools` list drives all storage provisioning. For each entry:
1. **Raw device path (`/dev/...`):** The playbook creates an LVM Volume Group and LVM Thin Pool on the specified device, then registers the resulting thin pool in Linstor.
2. **Existing thin pool path (`vg/lv`, e.g. `pve/data`):** LVM creation is skipped entirely. The existing thin pool is registered directly in Linstor without any modifications to the underlying storage.
After registering storage pools, the playbook creates one Linstor Resource Group per pool entry with a `place-count` equal to the total number of nodes in the inventory, then creates a Volume Group under it to make it available for VM disk provisioning.
## local-lvm Removal
By default, a fresh Proxmox install registers `local-lvm` pointing at `pve/data`. If any storage pool in your configuration targets `pve/data` — either directly or as a raw device that will end up there — the playbook automatically removes `local-lvm` via `pvesm remove local-lvm`.
This prevents operators from accidentally provisioning VMs to the underlying thin pool outside of Linstor, which would cause thin-pool exhaustion invisible to DRBD. The `local` storage entry (ISO images, backups at `/var/lib/vz`) is always retained.
## Proxmox Storage Backend
After Linstor storage is configured, each Resource Group is registered in the Proxmox storage subsystem:
```bash
pvesm add drbd ls-<rg_name> --controller <ha_vip> --resourcegroup <rg_name> --content images,rootdir
```
The controller address points to `ha_vip` so that VM operations always reach the active Linstor controller regardless of which node it is running on.
## HA Controller via drbd-reactor
The Linstor controller database is stored on a DRBD-replicated volume (`linstor_db`) rather than locally, enabling automatic failover:
1. A 200M DRBD resource (`linstor_db`) is spawned across all nodes from the HA storage pool.
2. The standalone `linstor-controller` systemd service is disabled on all nodes.
3. A `var-lib-linstor.mount` systemd unit is deployed — it mounts the DRBD device at `/var/lib/linstor` when promoted.
4. `/var/lib/linstor` is marked immutable with `chattr +i`. If the DRBD device fails to mount, the immutable flag prevents any writes to the local filesystem — eliminating the risk of a controller database split-brain.
5. A `drbd-reactor` promoter config is deployed at `/etc/drbd-reactor.d/linstor_db.toml`. It manages:
- Promoting `linstor_db` to Primary on one node
- Mounting `/var/lib/linstor`
- Bringing up the floating VIP via `ocf:heartbeat:IPaddr2`
- Starting `linstor-controller.service`
Failover is automatic: if the active node loses quorum or crashes, `drbd-reactor` on another node promotes the resource and brings up all services within seconds.
### Checking HA Status
```bash
drbd-reactorctl status linstor_db
```
Expected output on the active node:
```
Promoter: Resource linstor_db currently active on this node
● drbd-services@linstor_db.target
● ├─ drbd-promote@linstor_db.service
● ├─ ocf.rs@service_ip_linstor_db.service
● ├─ var-lib-linstor.mount
● └─ linstor-controller.service
```
---
[Previous: Architecture & Phase Breakdown](04-architecture.md) | [Index](../README.md) | [Next: Troubleshooting & FAQ](06-troubleshooting.md)