ZFS facts for kids

ZFS (which used to stand for Zettabyte File System) is a special kind of file system that also helps manage storage space. It was first created by Sun Microsystems in 2001 as part of their Solaris operating system. ZFS was made open source (meaning its code was free for anyone to use and change) in 2005 as part of OpenSolaris.

However, when Oracle Corporation bought Sun in 2009–2010, ZFS became closed source again. Because many people liked the open source version, a new project called OpenZFS was started in 2013. OpenZFS now helps manage and develop the open source ZFS code, which is used in many Unix-like computer systems.

How ZFS Works

Usually, managing computer storage involves two main parts:

• Volume management: This is about organizing physical storage devices like hard drives into bigger, logical units that the computer can see. Think of it like combining several small boxes into one big storage container.
• File system: This is about how files and data are actually stored and organized on those logical storage units. It's like deciding where to put each item inside your big storage container.

ZFS is unique because it does both of these jobs at the same time! It knows everything about your physical disks and how your files are stored on them. This complete control helps ZFS make sure your data is safe from errors, damage, or bit rot (when data slowly degrades over time). It constantly checks, confirms, and fixes things to keep your information secure.

ZFS also has a powerful feature called snapshots. A snapshot is like taking a quick picture of your data at a specific moment. You can take many snapshots without slowing down your computer. This is super useful if you make a mistake or if something goes wrong, because you can easily go back to an earlier version of your files. You can even create new, working copies of your files from these snapshots, which are called clones.

ZFS History

Early Development (2004-2010)

ZFS was designed by a team at Sun Microsystems, including Jeff Bonwick. They started working on it in 2001 and officially announced it in 2004. The code was added to OpenSolaris in 2005 and became part of Solaris 10 in 2006.

Sun Microsystems made ZFS open source in 2005. This meant that other operating systems like Linux, Mac OS X, and FreeBSD could use and adapt ZFS. The name "Zettabyte File System" came from its ability to store an incredibly huge amount of data – up to 256 quadrillion zettabytes!

Oracle and OpenZFS (2010-Present)

When Oracle Corporation bought Sun Microsystems in 2010, Oracle decided to make their version of ZFS closed source again. This meant the public could no longer freely access or change the code.

Because ZFS was so useful, many developers wanted to keep it open source. So, in 2013, the OpenZFS project was created. OpenZFS helps different groups work together on the main ZFS code, while each group can also add their own specific features to make ZFS work best with their systems.

Key Features

ZFS has many cool features that make it a very reliable and flexible storage system:

• Built for the Future: It's designed to store data for a very long time and can grow to handle huge amounts of information without losing anything.
• Self-Healing Data: ZFS uses special codes called checksums to check all your data. If it finds any errors or corruption, it can often fix them automatically, especially if you have multiple copies of your data.
• Multiple Copies: You can tell ZFS to store extra copies of important data or information about your files. This makes it even safer against corruption.
• Automatic Rollback: If something goes wrong, ZFS can sometimes automatically undo recent changes to your files, helping to prevent data loss.
• Smart Storage Management: ZFS handles different ways of storing data, like RAID (which combines multiple disks for speed or safety) and caching (using faster storage for frequently used data). It does this in a very smart way because it understands both the disks and the files.
• Snapshots and Backups: As mentioned, ZFS can take quick snapshots of your data. These snapshots can also be used to make efficient backups or copies of your data.
• Compression and Deduplication: ZFS can shrink your files to save space (compression) and avoid storing the same data multiple times (deduplication). Deduplication needs a lot of computer memory to work well.
• Easy Disk Swaps: If you move your ZFS storage to new hardware or reinstall your operating system, ZFS can still understand and access your data without problems. This is often not the case with other storage systems.
• Smart Caching: ZFS uses different levels of cache (fast memory) to speed up how quickly it can read and write data. It learns what data you use most often and keeps it in the fastest cache.
• Highly Customizable: You can adjust many settings in ZFS to make it work best for your specific needs.

Keeping Data Safe

One of the most important things about ZFS is how it protects your data. It's designed to stop "silent data corruption," which is when your data gets damaged without you knowing it. This can happen from things like power surges or tiny errors on your hard drive.

ZFS does this by using checksums. Imagine each piece of data has a unique fingerprint. ZFS stores this fingerprint separately from the data itself. When you try to read the data, ZFS checks its fingerprint. If it doesn't match, ZFS knows the data is bad. If you have multiple copies of your data (for example, if you're using mirroring), ZFS can then use a good copy to fix the bad one. This is called "self-healing."

RAID-Z: ZFS's Own RAID

To keep data safe, ZFS often needs to store multiple copies across different disks. It does this using its own built-in system called RAID-Z or by mirroring disks.

Why ZFS Prefers Direct Disk Access

Most computer systems use special hardware RAID controllers to manage multiple disks. But ZFS works best when it can talk directly to each hard drive. This is because ZFS has its own smart ways of organizing data, caching, and recovering from errors. If a hardware RAID controller gets in the way, it can actually make ZFS less efficient and less able to protect your data.

So, if you're using ZFS, it's usually better to connect your disks directly to your computer or use a simple adapter that doesn't try to do its own RAID management.

RAID-Z and Mirroring Explained

Instead of hardware RAID, ZFS uses "soft" RAID.

• Mirroring is like having an exact copy of your data on another disk. If one disk fails, you still have all your data on the mirrored disk.
• RAID-Z is similar to RAID-5 or RAID-6. It spreads your data and special "parity" information across multiple disks. This means if one (RAID-Z1), two (RAID-Z2), or even three (RAID-Z3) disks fail, ZFS can rebuild your lost data using the remaining information. RAID-Z is also smarter than traditional RAID because it can fix only the damaged parts of your data, making repairs much faster.

Checking and Fixing Data (Scrubbing)

Unlike other file systems that use a tool called `fsck` (which often requires you to stop using your files), ZFS has a built-in feature called scrub. You can run a scrub while you're still using your computer. It checks all your data and metadata (information about your files) for errors and fixes them using the extra copies or parity data. This process can take many hours for large storage systems, but it's very thorough and helps keep your data healthy.

Huge Storage Capacity

ZFS is a "128-bit" file system. This means it can handle an incredibly vast amount of data – so much that you'll probably never reach its limits! For example, a single ZFS storage pool can be as large as 256 quadrillion zebibytes. To give you an idea, that's like having billions of the largest hard drives available today.

Some of its theoretical limits include:

• Maximum size of one file: 16 exbibytes (a very, very large file!)
• Maximum number of files in a directory: 2⁴⁸ (a huge number!)
• Maximum size of a storage pool: 256 quadrillion zebibytes

Data Encryption

ZFS can encrypt your data, which means it scrambles your information so only people with the right key can read it. This adds an extra layer of security. You can set up encryption for different parts of your storage, and it works smoothly as you write and read files.

How ZFS Speeds Things Up

ZFS is designed to make reading and writing data as fast as possible. It spreads data across all your storage devices evenly to balance the workload. This means when you read data, different parts can be pulled from many disks at once, making it much quicker.

Smart Caching

ZFS uses different layers of fast memory (cache) to speed up how it handles data:

• ARC (Adaptive Replacement Cache): This is the main read cache and lives in your computer's main memory (RAM). ZFS tries to keep the most frequently used data here so it can be accessed super fast.
• L2ARC (Level 2 ARC): This is an optional second level of read cache, usually on fast solid-state drives (SSDs). It's used for data that's not quite "hot" enough for RAM but still accessed often.
• ZIL (ZFS Intent Log) / SLOG (Separate Log Device): These help speed up "synchronous writes" (when your computer needs to know data is safely saved before moving on). ZFS writes these important updates to a very fast device (the SLOG) first, then moves them to the main storage later. This makes your computer feel faster, especially for databases or virtual machines. If your computer loses power, the ZIL/SLOG ensures that no data that was confirmed as written is lost.

Copy-on-Write

ZFS uses a "copy-on-write" system. This means when you change a file, ZFS doesn't just overwrite the old data. Instead, it writes the new data to a fresh spot on the disk. Then, it updates the pointers to the new data. This is how ZFS can create snapshots so quickly, because the old data blocks are still there, ready to be accessed as a "picture" of the past.

Snapshots and Clones

Because of copy-on-write, ZFS can create "snapshots" instantly. A snapshot is a read-only copy of your file system at a specific moment. They don't take up much extra space because they only store the changes made since the snapshot was taken. You can use snapshots to go back to previous versions of files or even entire systems.

You can also create "clones" from snapshots. A clone is a working, editable copy of a snapshot. Both the original and the clone share the unchanged data blocks, saving space.

Sending Snapshots

ZFS can "send" snapshots to other storage pools, even on different computers over a network. This is very efficient because you can send only the changes between two snapshots, rather than the entire file system. This is great for making backups in another location.

Other Useful Features

• Variable Block Sizes: ZFS can use different block sizes (the chunks of data it stores) to be more efficient, especially when using compression.
• Easy File System Creation: Creating new file systems in ZFS is as simple as making a new folder.
• Deduplication: ZFS can find and remove duplicate copies of data, saving a lot of space. However, this feature needs a lot of computer memory to work well.

What ZFS Can't Do (Yet)

While ZFS is amazing, it has a few limitations:

• You generally can't easily shrink a ZFS storage pool by removing disks from a RAID-Z group. You can add more disks, but taking them away is harder. However, newer versions of OpenZFS are working on this.
• You can't directly add a single disk to an existing RAID-Z group to expand it. You have to create a new RAID-Z group and add it to the overall storage pool.

Getting Data Back

ZFS is designed to be self-healing, so it doesn't have a tool like `fsck` (which other file systems use to check and repair errors). If your storage pool was set up correctly with enough backup copies, ZFS should be able to fix itself.

However, if a ZFS pool gets very damaged (for example, if too many disks fail at once), it might not be able to "mount" (become accessible). In the past, it was hard to get data back from such a badly damaged pool.

But modern ZFS versions have improved a lot:

• Losing a caching device (like an SSD used for L2ARC) no longer causes the whole pool to fail.
• If a pool can't be mounted, ZFS can often find the most recent good version of your data and recover it, even if you lose a few very recent changes.
• Special tools exist that can help experts figure out why a pool isn't mounting and sometimes manually fix it.
• Newer OpenZFS features are making it even easier to diagnose and recover data from damaged pools, even if some parts are missing.

OpenZFS vs. Oracle ZFS

After Oracle bought Sun in 2010, they stopped making ZFS open source. This led to the creation of the illumos project and then OpenZFS in 2013.

Today, "Oracle ZFS" is a closed-source product developed by Oracle. "OpenZFS" is the open-source version, developed by a community of engineers from different companies and projects. Over time, OpenZFS has added many new features and changed much of the original code, making it quite different from Oracle's version.

Products Using ZFS

Many companies use ZFS in their products because of its powerful features:

• iXsystems uses ZFS in their FreeNAS (now TrueNAS CORE) and TrueNAS devices, which are popular for network storage.
• Netgear has used ZFS in some of its storage devices.
• rsync.net offers cloud storage where customers can use ZFS features.
• Proxmox VE and Ubuntu Linux also offer native support for ZFS.

See also

In Spanish: ZFS (sistema de archivos) para niños

• Comparison of file systems
• List of file systems
• Versioning file system – List of versioning file systems