I’m a little teapot 🫖

  • 1 Post
  • 166 Comments
Joined 1 year ago
cake
Cake day: September 27th, 2023

help-circle
  • I wrote snapshot hooks for Arch that fire before installing or upgrading packages and I have a simple shell alias that I can use to fire off a manual snapshot any time I need one. If a package breaks in an inconvenient way and can’t just be dowgraded back to function or I have some other time pressure I can just point my root partition at a clone of my most recent snapshot and reboot to roll back. I don’t usually bother rebooting into a cloned snapshot to test changes as I can just perform the same steps to roll back and the automated rolling snapshots mean I don’t need to baby anything to have the same protection.







  • Your firewall should take care of that, it’s pretty rare to be connected directly without one and by default any decent routing package will filter incoming traffic that’s not in the state tracking table. NAT isn’t designed for security, any security benefit it provides is a side effect rather than the intended purpose.

    Edit: check out ipv6 privacy extensions too, there are solutions there that can reduce info disclosure if that’s a concern. You can accomplish many of the same benefits of NAT with v6 features without the downsides that NAT brings.




  • The best way to keep an Arch daily driver stable is to use a CoW filesystem like btrfs or ZFS and use a pacman hook that takes a snapshot before package installation or updates. If you run an update or package install that breaks something in an inconvenient way 20 minutes before you have to be on a call using said system you just revert to the snapshot, reboot and you’re rolled back. I’ve run Arch as my daily driver like this for 4-5y now and can count the number of times I’ve needed to roll back like this on one hand (mostly I just don’t run updates when I’m under time pressure.) Arch really doesn’t break in a major way all that often once you’re over the first 6-9mo of the learning curve, ie: it usually stops breaking once you stop fucking around and poking things while you’re learning about them.

    I wouldn’t recommend Arch as a daily driver to someone who isn’t generally competent at linux admin and troubleshooting, it’s a handful for people who aren’t up on their skills yet. On the other hand it definitely will help you develop those skills, that just takes some time. A snapshot and rollback system gives you the freedom to break most things with a reliable way to roll back to a known good state afterwards.


  • Most enterprise drives are TLC these days, MLC just doesn’t provide the storage density that enterprises require anymore. I only mentioned MLC because you’ll occasionally find mSATA drives in the <=256GB range that use MLC. You have to check the datasheet for each model, look for endurance rated at 5DWPD or higher, those will typically be MLC or heavily over provisioned TLC. If you want enterprise drives with greater endurance than the usual 0.5 or 1 DWPD look for the over provisioned models with capacities like 400GB, 800GB, 1.6T or 3.2T. those are 512GB, 1TB, 2TB and 4TB raw capacity drives with a bunch of flash set aside for wear leveling purposes. You don’t often see 300GB, 600GB, 1.2T or 2.4T drives anymore but those are often very high endurance (write intensive, 10 DWPD or so) models.

    Check the datasheets for drives when you’re shopping and you can get a pretty good idea of what their durability is like, I usually buy 1 DWPD drives for write occasional bulk storage and 3+ DWPD for anything with a serious write workload. You can also help the drive controller a bit by running blkdiscard against the entire device before partitioning, then only partition and use ~80% of available space. The drive controller will typically grab free unused blocks and use them for wear leveling but only if they’ve been marked free (TRIMmed) and never allocated after. If you can’t find or can’t afford high endurance drives you can usually buy a larger lower endurance drive and over provision it in this way to extend its lifespan.

    (The last time MLC flash was really common was back in maybe 2014-2015, some of the older Samsung pro drives like the 850/860 pro were built using MLC. Those had legendary real world endurance, I think they’d get up to 10+PB written before actually failing. It’s a shame they didn’t have PLP because they would have made good budget array storage if they did.)


  • My approach to this has always been to buy one enclosure and validate it, then go buy like 8 more after thorough testing. Obviously don’t place an order for 10 units of an unknown tech item from AliExpress or you’re looking at a bad time. Look for enclosures that use known good chipsets and there’s not as much risk as you’re expecting. I have something like 8 msata enclosures here that work flawlessly and another half dozen sata+nvme rtl9210b enclosures that also work well.


  • Buy used Samsung mSata or m.2 2230 drives on fleaBay. Stick with Samsung and other well known brands with decent spec sheets and warranties, that’s the cheapest way to handle durable storage on a pi. USB enclosures are like $5-7 on AliExpress or fleaBay.

    Buy MLC drives if you need higher endurance (check the model no and look up the datasheet.) TLC will usually be fine for a few years, MLC will last a bit longer. If you’re killing drives faster than you expect buy larger (512 instead of 256GB), blkdiscard the entire device once it’s installed and then only partition 60-80% of it. Never touch the rest of the freed storage and the drive controller should be able to use those blocks for wear levelling to reduce the NAND wear rate.

    Edit: One heads up, I usually buy used drives from eBay because their buyer protection is top tier, if there’s anything wrong with the drive when it’s delivered or when I test it it goes right back for a refund. This makes buying blind viable thanks to an easy return policy.

    If you’re sourcing used drives somewhere else insist on seeing SMART data before purchasing and don’t buy heavily worn drives. Look at the drive model datasheet, find the warranted endurance of the drive (if it’s a 512GB drive rated for 1 DWPD over 3y that means the rated endurance is ~ 0.5T * 365 * 3 or roughly ~550TB written over 3y. Pass on buying drives approaching their rated endurance, try to buy lightly used drives wherever possible and you shouldn’t have problems with reliability.



  • Because you’re relying on compatibility between older Debian software (systemd, etc) and newer versions installed in the chroot. Things get weird quickly.

    Consider a nested privileged container instead (LXC or similar) and cross your fingers that Debian systemd and Arch systemd play nice.

    If the above fails just make a VM and pass through the GPU with GVT-g (otherwise pass through the entire GPU.)

    If all of that fails install Arch to a USB attached SSD or something.


  • If you’re using an Intel chip look into GVT-g and consider running Arch from a VM, that’ll be the closest thing to native.

    The unfortunate thing about running an Arch container from a Debian host is that you’re relying on an older kernel and an older systemd host side and I’ve found that often causes compatibility problems inside the Arch container. If you are very, very lucky Arch will just work inside the container, but IME that’s fairly rare as systemd often has breaking changes over several releases (and Arch tends to be at least several releases ahead of Debian.)


  • Buy external drives. Don’t run them in RAID, use one to store backups and plug it in once or twice a week to copy data to it.

    The secret to RAID is that it doesn’t buy you data protection, it buys you uptime to access data while a device in the array is failed. This is most valuable to businesses that can’t afford the downtime that recovery from a backup incurs. The most paranoid RAID will still fail sooner or later, due to hardware or software failure, and as a home user with a limited budget you’re far better off having one offline backup that you can use to recover data from once that happens.

    Backup only data you can’t afford to lose (eg: don’t backup downloaded data that can be replaced easily, like a game or movie collection) and your backups will be much more manageably sized and you won’t need to spend as much on your backup drive. If a backup disk is too much for your budget you can always exploit cloud backup plans, backblaze PC backup has no limit on the size of your backups and only charges something like ~$60/yr.

    Edit: It’s also worth thinking about what kind of data you’re storing and splitting that data across multiple devices if possible. If you’re storing bulk data where performance isn’t critical, like backups from other machines or a movie collection, you can pay a much lower price by buying a hard drive instead of flash. Even if only some of your data requires fast flash you can still use a cheaper HDD to store bulk data and buy a smaller flash drive for performance sensitive tasks. When I build NAS I split my data two pools, one bulk pool of HDDs and one much smaller fast pool comprised of flash storage. Put performance critical data on flash, put bulk storage on HDDs, this will allow you to spend less on bulk and still have fast storage performance for tasks that require it. A 512GB or 1TB SSD alongside a 4TB, 6TB or 8TB HDD is significantly cheaper than spending on a 4TB or 8TB SSD.

    Shop eBay for refurbished storage, it’ll be significantly cheaper than spending on brand new drives.


  • Btrfs, ZFS and ext4. My servers use ZFS, my client machines mostly use btrfs and I have a sprinkling of ext4 partitions for specific workloads. I’m all in on CoW filesystems for snapshots, send receive, transparent compression and reflinks. I like btrfs on client machines and SBCs because it’s easily available (baked into the kernel) and doesn’t require maintaining dkms or holding kernel versions until ZFS supports them and because snapshot handling and other filesystem admin tasks are simple and straightforward. I run ZFS wherever data integrity is important, eg: storage servers and backup targets, but largely prefer working with btrfs.