OpenZFS Office Hours with Matt Ahrens

Channel:OpenZFS · Length:1:25:24 · Published:2013-10-11

Watch on YouTube

Summary

An OpenZFS office hours session with ZFS co-creator Matt Ahrens answering questions on topics such as hole birth time, send/receive buffers, device removal, block pointer rewrite and performance with many snapshots.

Why HN recommends it

Cited as the authoritative explanation of why block pointer rewrite is so hard to implement in ZFS.

The above video is an explanation of a bunch of the barriers to implementing block pointer rewrites. — HN, 2014

Databases & storage · Operating systems & kernels

Subtitles (excerpt)

foreign so um I'm coming to you guys live uh from San Francisco I'm Matt Ahrens I I started the ZFS project with Jeff bonwick back in 2001 at Sun maker systems and I'm here today to answer questions about uh ZFS and open ZFS and anything you might want to talk about so uh looks like we have just a few people here uh extremely promptly um so take it away um Derek uh Karen or prakash anything that I can tell you about open ZFS Tubity has questions all right um then I will just start telling you about uh some ZFS stuff that I'm working on um so uh right now I'm working on a project with um Max uh Max Grossman who's another employee at delphix um he is stuck on a train right now or uh he would be joining us um but we're working on a project that he mostly implemented and the idea is that if you've ever used EFS send and receive say for backups or for remote replication then um uh We've noticed the problem with it a performance problem where um when you're doing a send of a stream that has lots of holes in it so for example uh this happens a lot with uh sending his evolves so we noticed this problem when we were sending uh zvolves for remote replication um from one machine to another so Z Vols hold other file systems and they start out

On Hacker News

On my system the root account is primarily for upgrades, but also storage administration! ZFS snapshots protects me against silly mistakes like strange permutations of `rm -rf ~/` in day to day operations. Not having root all the time protects me against `zfs destroy -r nagato/data/home/drbawb` which will nuke my home folder, 144 frequent snapshots (15 minute intervals per 36 hours), 14 daily sna… — drbawb, 2023-07
No, there is no defragmentation for ZFS, unfortunately. A way to get around that is to send the pool's content to another (fresh) ZFS pool, where it would be written sequentially. But for that you would need a set of drives of same (or larger) capacity. There are ideas on how one would do an actual defrag. They are generally based on a concept called block pointer rewrite, which Matt Ahrens once … — ploek, 2023-03
> Illumos has a way to expand pools ... ZFS creates a vdev inside the new larger vdev Huh? > IDK if that's in OpenZFS yet. The openzfs tree (on github) is virtually identical to illumos-gate (on github). > physical block addresses should never have been part of the ZFS Merkle hash tree, thus allowing physical addresses to change without having to rewrite every block from the root down. mahrens de… — raattgift, 2017-07
[VIDEO] The above video is an explanation of a bunch of the barriers to implementing block pointer rewrites. The conclusion is that it would make the code a lot more complicated and break a lot of the layering, and probably make addition of other new features a lot harder. Even a standalone offline rewriting tool wouldn't necessarily be accepted into the OpenZFS codebase because of the maintenanc… — wtallis, 2014-11

Related videos