93 lines
3.2 KiB
Markdown
93 lines
3.2 KiB
Markdown
# 06 - Troubleshooting & FAQ
|
|
|
|
## SSH: Host Key Verification Failed
|
|
|
|
If a node was reinstalled, its host key changes and SSH refuses to connect:
|
|
|
|
```
|
|
WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!
|
|
```
|
|
|
|
Remove the stale entry and re-scan:
|
|
|
|
```bash
|
|
ssh-keygen -R <ip>
|
|
ssh-keyscan <ip> >> ~/.ssh/known_hosts
|
|
```
|
|
|
|
## APT: SSL Certificate Verification Failed
|
|
|
|
If `apt update` fails with `certificate verify failed`, check whether your firewall is performing SSL inspection on the APT repository domains. The playbook uses `http://` for the Proxmox no-subscription repository (which does not support HTTPS) and `https://` for the enterprise and LINBIT repositories.
|
|
|
|
If your FortiGate or similar is intercepting HTTPS traffic to these domains, add them to the SSL inspection exemption list:
|
|
|
|
- `*.proxmox.com`
|
|
- `*.linbit.com`
|
|
- `packages.linbit.com`
|
|
|
|
## APT: LINBIT Keyring Download Times Out
|
|
|
|
The LINBIT keyring is downloaded via Python's `urllib`. On nodes where IPv6 is configured but not routed, the download stalls while waiting for the IPv6 connection attempt to time out before falling back to IPv4. Set `disable_ipv6: true` in `group_vars` to avoid this.
|
|
|
|
## LVM: Volume Group Creation Fails (Dirty Disk)
|
|
|
|
If the `Ensure Volume Groups exist for raw disks` task fails, the target device likely has existing partition tables or filesystem signatures. Set `wipe_linstor_disks: true` in `group_vars` and re-run. The playbook will run `wipefs -a` and `sgdisk --zap-all` before attempting LVM creation.
|
|
|
|
**Never set `wipe_linstor_disks: true` on a cluster that already has data.**
|
|
|
|
## Linstor: Nodes Not Appearing as Online
|
|
|
|
Check node status from Node 1:
|
|
|
|
```bash
|
|
linstor node list
|
|
```
|
|
|
|
If nodes show as `OFFLINE` or `CONNECTING`, verify:
|
|
1. The satellite service is running: `systemctl status linstor-satellite`
|
|
2. The private network (if configured) is reachable: `ping <priv_ip>`
|
|
3. Firewall allows port 3366/tcp (Linstor satellite default)
|
|
|
|
## Linstor HA: Controller Not Promoting
|
|
|
|
If `drbd-reactorctl status linstor_db` shows `currently active on <unknown>` with all services inactive:
|
|
|
|
```bash
|
|
systemctl reload drbd-reactor
|
|
```
|
|
|
|
If the TOML config was deployed but `drbd-reactor` was not reloaded (e.g. the service was already running before the playbook ran), it will not pick up the new config until reloaded. The playbook handles this automatically on subsequent runs via a handler, but a manual reload fixes it immediately.
|
|
|
|
Also verify the DRBD resource has at least one node with a disk in `UpToDate` state:
|
|
|
|
```bash
|
|
drbdadm status linstor_db
|
|
```
|
|
|
|
## Corosync: Quorum Loss
|
|
|
|
If clustering tasks hang on `pvecm status` or nodes cannot join, Corosync has lost quorum. In a 3-node cluster, 2 nodes must be online. To force a degraded single-node cluster to regain quorum temporarily (recovery only):
|
|
|
|
```bash
|
|
pvecm expected 1
|
|
```
|
|
|
|
## Complete Cluster Reset
|
|
|
|
To tear down Corosync and return a node to standalone state:
|
|
|
|
```bash
|
|
systemctl stop pve-cluster
|
|
pmxcfs -l
|
|
rm /etc/pve/corosync.conf
|
|
rm -rf /etc/corosync/*
|
|
killall pmxcfs
|
|
systemctl start pve-cluster
|
|
```
|
|
|
|
If DRBD resources are active on the node, an OS reinstall is faster and safer than manually tearing down DRBD metadata and LVM structures.
|
|
|
|
---
|
|
|
|
[Previous: Linstor & Storage Deep Dive](05-linstor-storage.md) | [Index](../README.md)
|