How to limit bandwidth of ZFS send/recv?

laskinforestlaskinforest
in Technical

Hey all,
I've been messing around with ZFS lately and it seems pretty cool. However, I can't seem to find a way to limit the bandwidth that is used for a ZFS send/recv.

I'm trying to send a ~400GB filesystem between two systems (sending is Ubuntu, receiving is FreeBSD): one at my house, and another on a server. I don't have a particularly fast connection (50 mbit/s), so I'm looking for a way to limit the bandwidth that ZFS uses so that my internet doesn't grind to a halt for almost a day.

I'm not in a rush to have the data transferred or anything, so I'd rather be able to use my internet than have it finish faster. I don't have a way of limiting the connection speed of a single host on my router, so a software solution on either the sending or receiving end would be highly preferred.

Any help would be greatly appreciated.

Thanked by (1)vimalware
Tagged:

Comments

  • MasonMason AdministratorOG

    Looks like zfs' built-in readlimit & sendlimit options are what you're looking for. I don't use zfs myself, but here's the commands:

    zfs set writelimit=10mb rpool/my/pool
zfs set readlimit=10mb rpool/my/pool

    Just set it to something low so that both of the limits combined won't eat up all your bandwich, i.e. readlimit=10mb,sendlimit=10mb. Adjust as needed or maybe even configure a cronjob that changes the limits when you're sleeping to get the max usage, then goes back down during your 'active' hours.

    Thanked by (3)vimalware laskinforest bdl

    Head Janitor @ LES • About • Rules • Support

  • seanhoseanho OG

    Doing the rate limiting directly in zfs is probably the best. In addition, you might investigate piping it through pv with the -L option.

    Thanked by (2)vimalware laskinforest
  • vimalwarevimalware OG

    And if you are using Proxmox pve-zsync for schedled/adhoc ZFS backups, it has the " --limit VALUE " option.

    https://pve.proxmox.com/wiki/PVE-zsync

    Thanked by (1)laskinforest
  • laskinforestlaskinforest

    I'm getting the following error when I try to set the limits (both on the Ubuntu and FreeBSD systems):

    cannot set property for 'poolname': invalid property 'writelimit'

    Any ideas for what could be going on? Ubuntu version is 18.04 and FreeBSD is 12.1.

  • comicomi OG
    edited July 2020

    @laskinforest said:
    I'm getting the following error when I try to set the limits (both on the Ubuntu and FreeBSD systems):

    cannot set property for 'poolname': invalid property 'writelimit'

    Any ideas for what could be going on? Ubuntu version is 18.04 and FreeBSD is 12.1.

    I think you need to set it on a filesystem, not entire pool

    EDIT: nope, you're right, it looks like it's not implemented in OpenZFS, only Oracle's version has that

    Thanked by (1)laskinforest
  • sureiamsureiam
    edited July 2020

    Not quite your question but why not instead just do overall nic bandwidth limiting either on your router or the operating system.

    https://vitux.com/how-to-limit-network-bandwidth-in-ubuntu/

    If you have a decent router or one that can take an open source flash you can even prioritize the mac address of your operating system so it uses 90% of your bandwidth but if anything else needs it then it drops to like 30%.

    That's the ideal route to take frankly otherwise just limiting upload/download speeds would be next.

    Thanked by (1)laskinforest
  • laskinforestlaskinforest

    @sureiam said:
    Not quite your question but why not instead just do overall nic bandwidth limiting either on your router or the operating system.

    https://vitux.com/how-to-limit-network-bandwidth-in-ubuntu/

    If you have a decent router or one that can take an open source flash you can even prioritize the mac address of your operating system so it uses 90% of your bandwidth but if anything else needs it then it drops to like 30%.

    That's the ideal route to take frankly otherwise just limiting upload/download speeds would be next.

    Yeah, I ended up just picking up a decent router and limiting it router-side. My old router couldn't do that, so I was wondering if there was a way of doing it without buying a new one. Thanks all for your suggestions and help.

    Thanked by (1)sureiam
  • sureiamsureiam

    @laskinforest said:

    @sureiam said:
    Not quite your question but why not instead just do overall nic bandwidth limiting either on your router or the operating system.

    https://vitux.com/how-to-limit-network-bandwidth-in-ubuntu/

    If you have a decent router or one that can take an open source flash you can even prioritize the mac address of your operating system so it uses 90% of your bandwidth but if anything else needs it then it drops to like 30%.

    That's the ideal route to take frankly otherwise just limiting upload/download speeds would be next.

    Yeah, I ended up just picking up a decent router and limiting it router-side. My old router couldn't do that, so I was wondering if there was a way of doing it without buying a new one. Thanks all for your suggestions and help.

    Yup that's the best route frankly. Silly thing to say but make sure you set the device to a static dhcp address on your router and also to do the rate limiting via mac address if possible vs IP

    Thanked by (1)laskinforest
Sign In or Register to comment.