cross-posted from: https://lemmy.cloudhub.social/post/2392
Figured we’d start this community off with a question about what you’re running in your homelab!
This could be anything from hardware to software to things your running in the cloud (#cloudlab).
Hardware and diagram pics are always welcome!
deleted by creator
And underrated.
I have a pretty modest setup. This is just what’s in or on my cabinet rack.
- Old two bay NAS
- New five drive bay server I’m replacing the old NAS with and running local stable diffusion and language models on. I managed to fit my old nVidia 3070 in the rack mount case. There’s no way a card the size of a 3090 would fit
- Some raspberry Pi’s
- Rack mount firewall
- Old Acer monitor, keyboard
- Dumb PDU and an old battery backup that I replaced the batteries on
- An old 802.11ac WiFi router set up as just a WAP, dedicated for home automation
Plan is to set up something like open stack but right now it’s just running unmanaged (orchestrated?) docker containers. I recently learned about ansible so may just automate the docker containers instead of figuring out open stack.
You fit a 3070 in a 5-bay NAS?? That’s impressive! I haven’t done much with ML, but it is a very interesting field of work. I’ve seen people do some pretty crazy things with it!
Ansible is nice, but have you heard of Terraform? Or, if you prefer programming/scripting as opposed to HCL/YAML, there is also Pulumi with lets you use terraform via a few different programming languages. (Ansible is nice though, I used to use it all the time in my lab, and it just works)
System76 Meerkat with attached external drive Unifi USG/Unifi APs/switches RaspberryPI/PiHole Emby Nextcloud Gitea Various simple websites
- My Raspberry Pi running Alpine, workint as a
dust collectorhome server - My Ryzen 5625U(from the top of my head) laptop which I use for light gaming and work mostly. Runs Artix Linux
- My beloved Ryzen 3 1200, RX 580, 2 1TB SSDs + 1 240GB SSD + 1 TB HDD. Also runs Artix Linux
- My Raspberry Pi running Alpine, workint as a
Hi! I’m Michael and this is my first lemmyverse post!
An old Lenovo thinkstation with 128Gb RAM, 512Gb SSD (x2), 4Tb SATA (x2) and 2Tb SATA for ISOs and backups. Running proxmox with VMs (Windows Server 2022, Home Assistant, Win 11 RDP jumpstation, OPNSense firewall, unifi controller and a Linux general purpose server). I have a dedicated server also running proxmox with a webserver, monitoring server (openitcockpit), meshcentral server.
Raspberry pi 4 as a backup and motioneye server in my garage.
A couple of other raspberry pi 4s doing things… Including 2 at my caravan running HA, Plex and general stuff.
Intel nuc
- homeassistant
- mqtt
- rtl433
- piper
- portainer
- zigbee2mqtt
- esphome
- calibre
- jellyfin
- doods
- pihole
- adguard
- valheim and other game servers Synology nas
- caldav
- redundant pihole
- files hosting
- unificontroller Older thin client
- opnsense with wireguard Unifi Switches and APs
Nice list! I’m curious, why are you running 2 pi-hole and an adguard instance?
(I also run 2 pi-hole instances for redundancy)
I have 3 vlans and have 1 blocker for each…was too lazy to configure rules per ip adress.
I’ve moved to technitium DNS nowadays. I found that it works better for me then AGH.
Box I built around a AMD Ryzen 7 3800X, running Ubuntu 22.04 and a handful of qemu VMs (owncloud, pihole, checkmk, etc…) A hand-me-down qnap I keep threatening to put truenas on but haven’t yet. A couple libre computer (pi alternative) boards. A couple tp-link managed switches.
On my to-do list are to deploy an old Dell mini as an OpnSense box to replace my router.
Has anyone tried running a Lemmy instance on theirs? I know it wouldn’t be a good idea to run one for public use, but I’m curious if anyone has tried just for fun.
I’m thinking about moving my single-user instance onto my lab from DO. Either that or moving to a managed Kubernetes cluster in the cloud (that is prohibitively expensive though)
How did you get it working on DigitalOcean? I tried that and it was such a struggle.
Lemmy? Had to patch the docker config (pushed a patch to the main and docs repos already!)
Oh awesome! I’ll try again. Thanks!
You might have to check those repos, I don’t know if the site has been updated.
It hasn’t yet, but I see the changes on the repos.
- 3 used MSFF PCs (i5, kingston SSDs, 24GB of ram each). All running proxmox, set up as a cluster.
- 1x Raspberry Pi 4 8GB. Running ubuntu.
- 1x Vultr 2vCPU/4GB RAM instance.
I’ve got a small kubernetes cluster set up using Talos with 3 controlplane / 3 workers in VMs on the proxmox nodes. The vultr node is also running Talos and attached to the same cluster. Their KubeSpan feature is pretty neat, automatic full mesh wireguard between all cluster nodes.
Traffic inside the cluster flows seamlessly between all nodes, and I can even use it as sort of a proxy server using Cilium’s Egress Gateway function.Meanwhile my Pi4 is running k3s, to host a few services needed to operate the main cluster, such as the Harbor registry operating as a cache and a zigbee2mqtt instance because I have a raspbee2 for a zigbee adapter.
The main reason I’m using K3S even on the single node Pi is because I very much like using flux to manage the deployments on the servers.
Network wise, I’ve got a USG-3P, one of the newer compact 16 port POE switch. And a pair of UAP-AC-LITE for APs.
Maybe one day I’ll get around to switching the USG for something a little more capable. And maybe capable of doing IPS/IDS on my 500M/100M internet connection. But no idea what kind of specs I’d need for that.Would also like a NAS but… eh… Maybe I’ll just see if i can add more storage to the proxmox nodes and expand the ceph cluster or something.
Actually. Now that I think of it, I should probably diagram that out hmm. Anyone know any good tools for making that?
draw.io is one, I’ve started using LucidChart (personally) and https://d2lang.com at work for process diagrams.
This sounds a lot like my old cluster config (I stepped away from the lab for a few months and forgot how it works, so started over lmao), but basically it would spin up a talos cluster on proxmox using terraform, and then bootstrap FluxCD and the rest of the software would be setup using that. It was a pretty slick system.
And seriously, Talos Linux is really, really, nice. If I ever manage to mess up a kubernetes node (which has happened a few times when I was messing around), I just wipe it, reboot it from the ISO, and reprovision it with the machine configuration.
Talos is a great OS! I just wish there was some way to get the IPs from DHCP via Proxmox so I could automate it with terraform.
Currently running a docker environment from a laptop with the following:
Firefly III - For budgeting
Seafile - For file sync. Was using OneDrive, but since it’s not supported by Linux went with Seafile. Works great!
Keycloak - SSO
Cloudflare Tunnel - For connection to my services from outside without needing to forward ports, and to enforce SSO for platforms that don’t support it.
PHP Apache - Hosting a few small websites
- Little servers - 3 x Pentium D-1508s w/32GB RAM, 2 x 400GB SSDs
- Big server - 1 x Dual Xeon E5-2650L v3 w 128GB RAM, 2 x 100GB SSDs, 2 x 400GB SSDs, 2 x 800GB SSDs, 8 x 4TB HDDs
- Desktop - 1 x Ryzen 5800X3D w 32GB RAM, 1 x 2TB NVMe SSD, RTX 4080
- Cheap TP-Link 10Gbps switch
- Proxmox running across the servers, the 400GB SSDs are running Ceph, everything else in ZFS
- VyOS in a VM for routing etc…, 2Gbps symmetric internet
- Mostly LXC at present, in the process of migrating that to Hashicorp Nomad (running inside VMs) backed by Ceph
On the big server, what do you use the assortment of SSDs for? I get specifically having a good chunk of solid state storage, but im wondering if you’re like me and just acquired them over time, or if there’s a specific purpose in mind.
Mostly over time - OS on the pair of 100s, the 800s were for containers/VMs - this use is moving over to CephFS though - the three smaller boxes are a recent addition.
I used to run 2 hp proliant servers. One Proxmox and one truenas. Now it’s just one old AMD PC running Ubuntu server with some dockers. My electric bill is thanking me. About to move it all to an Intel NUC and downsize even further.
Intel NUC with a hard drive for local stuff (*arrs, jellyfin), but nowadays because I plan to go back to full-time motorhoming I fire up stuff on DO, hetzner, AWS, GCS, etc as required. At the moment just a Lemmy and general purpose instance, but I do pop up the odd gameserver I’ve dockerized on one of these services while playing with friends
Awesome! Yeah, my instances are currently running on DO, but it’s pretty expensive hosting in the cloud when you have a lab at home. My internet here isn’t very good though, that’s the main thing stopping me from moving them on-prem.
Joe’s datacenter & hetzner server auctions are good deals if you’ve got bad internet and want to run your own multiple smaller VMs! Depending on latency in the case of hetzner.
But yeah, hosting at home is always great. I did it for years, but electricity prices began creeping up and I got tired of the maintenance
Yeah, that’s true, they do have pretty good prices. I like DO though because it’s where I started and they have a DC not too far from me, so latency is very low.
It’s also nice to pay for not having to deal with the hardware, and to also have the hidden costs go away (ie, electricity)
That’s true! Those do add up over time.
I’d love to go full cloud-native with a kubernetes cluster, but I can’t justify the $100+ a month for a reasonable cluster :(
That’s my disappointment as well! I’ve done k3s on a droplet, and it was nice, but I’d like to handover the control plane to a cloud provider when I’m experimenting without burning my wallet.
For sure, then you just have to worry about deploying apps. Seems a lot easier for testing.
I think vultr is actually cheaper then DO though.
My main machine is an Optiplex 7070 micro (i5 8th gen, 16gb ram, 500gb SSD + 4TB hdd). I also have a pi 3 + 4tb hdd for backups and a pi 4 for wireguard. I have a few other SFF computers, but I don’t have a use for them at the moment.
For services, I host many of the popular ones (nextcloud, portainer, paperless, etc.), but here’s 3 I haven’t seen mentioned a lot:
- komga (ebook reader, works well with tachyiomi on my tablet)
- kitchenowl (recipes and meal scheduling)
- calckey (activitypub server)
Noice, I’ve been meaning to setup something like paperless! Calckey looks like a good solution/alternative to Mastodon with an interesting user interface.
I haven’t heard of the other two, but I’ll definately check out kitchenowl, could use some more meal planning!
deleted by creator
