cloudxtreme/zfs-crypto

ZFS On Linux with crypto patches

Welcome to the unofficial zfs-crypto branch.

To make it clear, this branch has nothing to do with Sun, Oracle, ZFSOnLinux, OpenSolaris, IllumOS, OpenIndiana, SmartOS, FreeBSD etc.

If you run a legacy pool version=30, this branch will let you import and upgrade your pool to the standard pool version=5000, and it will set feature@encryption for any filesystem using encryption.

It is to aid those who happen to use zfs-crypto with pool version=30 for the short window that it was available. Before the feature@ pool version became standard, or when importing pools from Solaris.

There are new files,

zcrypt.c zcrypt.h zio_crypt.c zio_crypt.h dsl_crypto.c dsl_crypto.h libzfs_crypto.c zcrypt_common.c

which are kept "as is" as much as possible, including (possibly irrelevant) headers.

The crypto/api/ header files are from OpenSolaris.

The crypto/api implementation is brand new, and supports "bare minimum" features as needed by ZFS only.

Current support is in BETA.

Importing a Solaris pool can be done using: Solaris: zpool create -o version=30 -O version=5 thepool $devices... Solaris: zfs create -o encryption=aes-256-ccm thepool/secure Linux: zpool import -N thepool Linux: zpool upgrade thepool Linux: zfs mount thepool/secure

• MACs are in use, but compute_mac() is empty, not called?

• All "// FIXME" should be inspected. In particular, known areas which differ are PROP_ALIAS, PROP_INHERIT, crypto vs userquota,

• Removed KEY method pkcs11.

Example 1: Ask for password.

# zfs create -o encryption=aes-256-gcm mypool/BOOM
  Enter passphrase for 'mypool/BOOM':
  Enter again:
  kernel: [11266.250594] spl-crypto: Cipher test 'CKM_AES_CCM' -> 'sun-ccm(aes)' successful.
# zfs list
  NAME          USED  AVAIL  REFER  MOUNTPOINT
  mypool        142K   984M    31K  /mypool
  mypool/BOOM    31K   984M    31K  /mypool/BOOM

# zpool get all mypool

mypool  feature@async_destroy  enabled                local
mypool  feature@encryption     active                 local

Example 2: Using a raw key file

# dd if=/dev/urandom bs=32 count=1 > /out.key
1+0 records in
1+0 records out
32 bytes (32 B) copied, 4.6033e-05 s, 695 kB/s
# zfs create -o encryption=aes-256-gcm -o keysource=raw,file://`pwd`/out.key mypool/BOOM
# zfs get mountpoint,checksum,compression,copies,dedup,encryption,keysource,keystatus mypool/BOOM
NAME         PROPERTY     VALUE                SOURCE
mypool/BOOM  mountpoint   /mypool/BOOM         default
mypool/BOOM  checksum     sha256-mac           local
mypool/BOOM  compression  off                  default
mypool/BOOM  copies       1                    default
mypool/BOOM  dedup        off                  default
mypool/BOOM  encryption   aes-256-gcm          local
mypool/BOOM  keysource    raw,file:///out.key  local
mypool/BOOM  keystatus    available            -

Example 3: Using a hex key file

# dd if=/dev/urandom bs=32 count=1 | od -A n -v -t x1 | tr -d ' \n' > /out.key
1+0 records in
1+0 records out
32 bytes (32 B) copied, 1.2811e-05 s, 2.5 MB/s
# zfs create -o encryption=aes-256-gcm -o keysource=hex,file:///out.key mypool/BOOM

====================================================================== striped, unencrypted

$ tiotest -t8 -f 200 -d /striped_zpool/
Tiotest results for 8 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        1600 MBs |    3.2 s | 502.596 MB/s |  37.4 %  | 1290.1 % |
| Random Write   31 MBs |    0.3 s | 120.682 MB/s |   0.0 %  | 903.7 % |
| Read         1600 MBs |    0.3 s | 4682.758 MB/s | 222.4 %  | 5104.2 % |
| Random Read    31 MBs |    0.0 s | 4521.777 MB/s |   0.0 %  | 5787.9 % |
`----------------------------------------------------------------------'
Tiotest latency results:
,-------------------------------------------------------------------------.
| Item         | Average latency | Maximum latency | % >2 sec | % >10 sec |
+--------------+-----------------+-----------------+----------+-----------+
| Write        |        0.015 ms |        8.793 ms |  0.00000 |   0.00000 |
| Random Write |        0.026 ms |        7.953 ms |  0.00000 |   0.00000 |
| Read         |        0.006 ms |        9.339 ms |  0.00000 |   0.00000 |
| Random Read  |        0.007 ms |        0.030 ms |  0.00000 |   0.00000 |
|--------------+-----------------+-----------------+----------+-----------|
| Total        |        0.011 ms |        9.339 ms |  0.00000 |   0.00000 |
`--------------+-----------------+-----------------+----------+-----------'

====================================================================== zfs-crypt aes-256-ccm

$ tiotest -t8 -f 200 -d /striped_zpool/fs/
Tiotest results for 8 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        1600 MBs |    7.4 s | 216.343 MB/s |   5.4 %  | 742.1 % |
| Random Write   31 MBs |    0.2 s | 180.906 MB/s |  92.6 %  | 775.7 % |
| Read         1600 MBs |    0.3 s | 5341.238 MB/s |  73.4 %  | 6276.0 % |
| Random Read    31 MBs |    0.0 s | 4783.407 MB/s | 4898.2 %  | 1224.6 % |
`----------------------------------------------------------------------'
Tiotest latency results:
,-------------------------------------------------------------------------.
| Item         | Average latency | Maximum latency | % >2 sec | % >10 sec |
+--------------+-----------------+-----------------+----------+-----------+
| Write        |        0.026 ms |      835.720 ms |  0.00000 |   0.00000 |
| Random Write |        0.021 ms |        0.087 ms |  0.00000 |   0.00000 |
| Read         |        0.006 ms |        0.245 ms |  0.00000 |   0.00000 |
| Random Read  |        0.006 ms |        0.025 ms |  0.00000 |   0.00000 |
|--------------+-----------------+-----------------+----------+-----------|
| Total        |        0.016 ms |      835.720 ms |  0.00000 |   0.00000 |
`--------------+-----------------+-----------------+----------+-----------'

====================================================================== LUKS volumes with plain ZFS

dmcrypt/luks cipher/keysize: aes-xts-plain64, 512

tiotest -t 8 -f 200 -d /striped_crypt/
Tiotest results for 8 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        1600 MBs |    3.4 s | 477.243 MB/s |  11.9 %  | 1169.2 % |
| Random Write   31 MBs |    0.5 s |  68.848 MB/s |  52.9 %  | 332.7 % |
| Read         1600 MBs |    0.3 s | 4999.844 MB/s | 318.7 %  | 5899.8 % |
| Random Read    31 MBs |    0.0 s | 4510.030 MB/s | 5051.2 %  | 1154.6 % |
`----------------------------------------------------------------------'
Tiotest latency results:
,-------------------------------------------------------------------------.
| Item         | Average latency | Maximum latency | % >2 sec | % >10 sec |
+--------------+-----------------+-----------------+----------+-----------+
| Write        |        0.031 ms |       69.994 ms |  0.00000 |   0.00000 |
| Random Write |        0.066 ms |       35.496 ms |  0.00000 |   0.00000 |
| Read         |        0.006 ms |       15.828 ms |  0.00000 |   0.00000 |
| Random Read  |        0.007 ms |        0.030 ms |  0.00000 |   0.00000 |
|--------------+-----------------+-----------------+----------+-----------|
| Total        |        0.019 ms |       69.994 ms |  0.00000 |   0.00000 |
`--------------+-----------------+-----------------+----------+-----------'

dmcrypt/luks cipher/keysize: aes-cbc-essiv:sha256, 256

tiotest -t 8 -f 200 -d /striped_crypt/
Tiotest results for 8 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write        1600 MBs |    3.4 s | 467.855 MB/s |  37.4 %  | 1121.4 % |
| Random Write   31 MBs |    0.3 s | 105.175 MB/s |  80.8 %  | 518.3 % |
| Read         1600 MBs |    0.3 s | 5061.753 MB/s | 126.5 %  | 6150.0 % |
| Random Read    31 MBs |    0.0 s | 4566.711 MB/s |   0.0 %  | 5845.4 % |
`----------------------------------------------------------------------'
Tiotest latency results:
,-------------------------------------------------------------------------.
| Item         | Average latency | Maximum latency | % >2 sec | % >10 sec |
+--------------+-----------------+-----------------+----------+-----------+
| Write        |        0.028 ms |       59.958 ms |  0.00000 |   0.00000 |
| Random Write |        0.070 ms |       27.873 ms |  0.00000 |   0.00000 |
| Read         |        0.006 ms |        0.168 ms |  0.00000 |   0.00000 |
| Random Read  |        0.007 ms |        0.030 ms |  0.00000 |   0.00000 |
|--------------+-----------------+-----------------+----------+-----------|
| Total        |        0.017 ms |       59.958 ms |  0.00000 |   0.00000 |
`--------------+-----------------+-----------------+----------+-----------'

zfs/rogue