In the next ~6 months I’m going to entirely overhaul my setup. Today I have a NUC6i3 running Home Assistant OS, and a NUC8i7 running OpenMediaVault with all the usual suspects via Docker.

I want to upgrade hardware significantly, partially because I’d like to bring in some local LLM. Nothing crazy, 1-8B models hitting 50tps would make me happy. But even that is going to mean a beefy machine compared to today, which will be nice for everything else too of course.

I’m still all over the place on hardware, part of what I’m trying to decide is whether to go with a single machine for everything or keep them separate.

Idea 1 is a beefy machine and Proxmox with HA in a VM, OMV or TrueNAS in another, and maybe a 3rd straight Debian to separate all the Docker stuff. But I don’t know if I want to add the complexity.

Idea 2 would be beefy machine for straight OMV/TrueNAS and run most stuff there, and then just move HA over to the existing i7 for more breathing room (mostly for Frigate, which could also separate to other machine I guess).

I hear a lot of great things about Proxmox, but I’m not sold that it’s worth the new complexity for me. And keeping HA (which is “critical” compared to everything else) separated feels like a smart choice. But keeping it on aging hardware diminishes that anyway, so I don’t know.

Just wanting to hear various opinions I guess.

  • SaintWacko@slrpnk.netEnglish
    1·
    11 days ago

    I will always recommend Proxmox, not just because it’s really easy to add more stuff, but because it’s really safe to tinker with. You take a snapshot, start messing around, and if you break something you just revert to the snapshot

  • notfromhere@lemmy.mlEnglish
    1·
    11 days ago

    I’m running Proxmox and hate it. I still recommend it for what you are trying to do. I think it would work quite nicely. Three of my four nodes have llama.cpp VMs hosting OpenAI-compatible LLM endpoints (llama-server) and I run Claude Code against that using a simple translation proxy.

    Proxmox is very opinionated on certain aspects and I much prefer bare metal k8s for my needs.

  • TunaLobster@lemmy.worldEnglish
    1·
    10 days ago

    I did it purely so I could fully back up my server VM and move it to new hardware when I wanted to upgrade. I just have to install Proxmox, attach the NAS, and pull the VM backup. And just like that everything is back to running just as it was before the upgrade! Now just faster and more energy efficient!

  • jubilationtcornpone@sh.itjust.worksEnglish
    1·
    11 days ago

    I use Proxmox for Work and Hyper-V at home. Looking forward to retiring my old Hyper-V host and replace it with Proxmox because Hyper-V is a pain.

    Virtualization really helps with reliability. In particular, by allowing you to quickly take snapshots before doing anything destructive and by streamlining backup and recovery.

  • irmadlad@lemmy.worldEnglish
    1·
    9 days ago

    Best thing to do is give it a go and see what shakes out OP. I absolutely love both my Proxmox boxes. In my humble opinion, Proxmox was an easier set up, and the possibilities are endless really. It’s a solid freemium product. Couple it with the extensive Helper Scripts, and Jack’s a doughnut, Bob’s your uncle.

  • muusemuuse@sh.itjust.worksEnglish
    1·
    10 days ago

    Do you need clusters that can failure ver from one machine to another? Is yes, proxmox is good. If no, there are less complex options.