把 LXD Container 放在已經建立的 ZFS Pool 裡面

趁著最近有空,筆記一下學到的東西


原理

這次其實還蠻簡單的,請容我偷懶一下

\   zfsX z_voX1   /
 \---------------/
  |   zfs_pool  |
 /---------------\
/ sda sdb ... sdx \

zfs 與眾不同的地方就在 pool 這東西,它讓原本物理上各個獨立運作的磁碟,變成一個邏輯上的儲存池,而一個儲存池又可以再切分成不同的資料集(dataset,或者說是 zfs 這邊的 fs 做檔案統理解),各自有不同的快取、壓縮等效能設定。甚至可以設定 volume 出來當作一個區塊裝置(block device,邏輯上的硬碟)

不過這只是一半而已,我們都知道 LXD 是基於 chroot 的概念,創造出來的技術,而 zfs 在這個時候碰巧又提供了 snapshot(快照)的功能,可以很有效率的提供差異備份(CoW )的功能,自然成為首選!

NAME                                            USED  AVAIL     REFER  MOUNTPOINT
lxd_pool                                       1.29G  29.2G       24K  none
lxd_pool/containers                            1.28G  29.2G       24K  none
lxd_pool/containers/WebServ                    1.28G  29.2G     1020M  /var/snap/lxd...
lxd_pool/containers/WebServ@snapshot-20191213   137M      -      930M  -
lxd_pool/containers/WebServ@snapshot-20200121  93.5M      -      935M  -

▲ 範例,這個 zpool 裡面有一個 WebServ,和兩個 snapshot

動手

首先看一下目前系統裡面有哪些東西

play_pc@lxd-server:~$ sudo zpool status
  pool: mypool
 state: ONLINE
  scan: none requested
config:

	NAME        STATE     READ WRITE CKSUM
	mypool      ONLINE       0     0     0
	  vdb       ONLINE       0     0     0

errors: No known data errors

play_pc@lxd-server:~$ zfs list
NAME     USED  AVAIL  REFER  MOUNTPOINT
mypool    84K  19.3G    24K  /mypool

根據上面 2 個指令的結果,我們可以知道目前有一個 pool 叫做 mypool,掛在 /mypool,沒有特別設定 dataset

所以我們接下來要做的,就是新增一個 dataset,名子就叫做... lxd 好了

sudo zfs create mypool/lxd

接下來請執行 lxd init 初始化

play_pc@lxd-server:~$ sudo lxd init
Would you like to use LXD clustering? (yes/no) [default=no]: no
Do you want to configure a new storage pool? (yes/no) [default=yes]:
Name of the new storage pool [default=default]: lxd_pool
Name of the storage backend to use (btrfs, dir, lvm, zfs) [default=zfs]: zfs
Create a new ZFS pool? (yes/no) [default=yes]: no
Name of the existing ZFS pool or dataset: mypool/lxd
Would you like to connect to a MAAS server? (yes/no) [default=no]: no
Would you like to create a new local network bridge? (yes/no) [default=yes]: yes
What should the new bridge be called? [default=lxdbr0]:
What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]:
What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]:
Would you like LXD to be available over the network? (yes/no) [default=no]:
Would you like stale cached images to be updated automatically? (yes/no) [default=yes]
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]:

基本上都是直接按 Enter 就好,但是有幾個特別重要的東西要注意一下:

play_pc@lxd-server:~$ lxc storage list
+----------+-------------+--------+------------+---------+
|   NAME   | DESCRIPTION | DRIVER |   SOURCE   | USED BY |
+----------+-------------+--------+------------+---------+
| lxd_pool |             | zfs    | mypool/lxd | 4       |
+----------+-------------+--------+------------+---------+

▲ 用指令確認一下儲存媒體,沒問題!

試著建立一個 lxc

play_pc@lxd-server:~$ lxc launch ubuntu:18.04
Creating the container
Container name is: trusty-mammal            
Starting trusty-mammal

Yup! 正常運作

play_pc@lxd-server:~$ lxc list
+---------------+---------+-------------------+-----------------------------------------------+------------+-----------+
|     NAME      |  STATE  |       IPV4        |                     IPV6                      |    TYPE    | SNAPSHOTS |
+---------------+---------+-------------------+-----------------------------------------------+------------+-----------+
| trusty-mammal | RUNNING | 10.21.7.20 (eth0) | fd42:9048:15e0:c172:216:3eff:fe28:fbc7 (eth0) | PERSISTENT | 0         |
+---------------+---------+-------------------+-----------------------------------------------+------------+-----------+

對比

讓我們來駔個實驗,看看一般使用單獨 zpool 的設定,和使用 dataset 作為 backend 時會有什麼不同吧!

play_pc@lxd-server:~$ lxc list
+---------------+---------+-------------------+-----------------------------------------------+------------+-----------+
|     NAME      |  STATE  |       IPV4        |                     IPV6                      |    TYPE    | SNAPSHOTS |
+---------------+---------+-------------------+-----------------------------------------------+------------+-----------+
| trusty-mammal | RUNNING | 10.21.7.20 (eth0) | fd42:9048:15e0:c172:216:3eff:fe28:fbc7 (eth0) | PERSISTENT | 1         |
+---------------+---------+-------------------+-----------------------------------------------+------------+-----------+

▲ 單獨 zpool / dataset

play_pc@lxd-svr:~$ lxc list
+-------------------+---------+----------------------+-----------------------------------------------+------------+-----------+
|       NAME        |  STATE  |         IPV4         |                     IPV6                      |    TYPE    | SNAPSHOTS |
+-------------------+---------+----------------------+-----------------------------------------------+------------+-----------+
| teaching-crayfish | RUNNING | 10.213.230.63 (eth0) | fd42:930f:f22a:5da4:216:3eff:fe11:3536 (eth0) | PERSISTENT | 1         |
+-------------------+---------+----------------------+-----------------------------------------------+------------+-----------+

因為名子是自動生成的(就像 Docker 一樣),請不要太在意

接下來是 zfs 的部份

play_pc@lxd-server:~$ sudo zfs list -t all
NAME                                                                                          USED  AVAIL  REFER  MOUNTPOINT
mypool                                                                                        920M  18.4G    24K  /mypool
mypool/lxd                                                                                    919M  18.4G    24K  none
mypool/lxd/containers                                                                         256M  18.4G    24K  none
mypool/lxd/containers/trusty-mammal                                                           256M  18.4G   840M  /var/lib/lxd/storage-pools/lxd_pool/containers/trusty-mammal
mypool/lxd/containers/trusty-mammal@snapshot-20200128                                        44.3M      -   841M  -
mypool/lxd/custom                                                                              24K  18.4G    24K  none
mypool/lxd/deleted                                                                             24K  18.4G    24K  none
mypool/lxd/images                                                                             663M  18.4G    24K  none
mypool/lxd/images/979ff60086ca9fb1c93d3131f8eca291820a524f0bd0b7d621a3c2f5f41ef185            663M  18.4G   663M  none
mypool/lxd/images/979ff60086ca9fb1c93d3131f8eca291820a524f0bd0b7d621a3c2f5f41ef185@readonly     0B      -   663M  -
mypool/lxd/snapshots                                                                           48K  18.4G    24K  none
mypool/lxd/snapshots/trusty-mammal                                                             24K  18.4G    24K  none

▲ 單獨 zpool / dataset

play_pc@lxd-svr:~$ sudo zfs list -t all
NAME                                                                                        USED  AVAIL  REFER  MOUNTPOINT
lxd_pool                                                                                    349M  18.9G    24K  none
lxd_pool/containers                                                                        7.00M  18.9G    24K  none
lxd_pool/containers/teaching-crayfish                                                      6.98M  18.9G   342M  /var/lib/lxd/storage-pools/lxd_pool/containers/teaching-crayfish
lxd_pool/containers/teaching-crayfish@snapshot-20200128                                      36K      -   342M  -
lxd_pool/custom                                                                              24K  18.9G    24K  none
lxd_pool/deleted                                                                             24K  18.9G    24K  none
lxd_pool/images                                                                             342M  18.9G    24K  none
lxd_pool/images/979ff60086ca9fb1c93d3131f8eca291820a524f0bd0b7d621a3c2f5f41ef185            342M  18.9G   342M  none
lxd_pool/images/979ff60086ca9fb1c93d3131f8eca291820a524f0bd0b7d621a3c2f5f41ef185@readonly     0B      -   342M  -
lxd_pool/snapshots                                                                           48K  18.9G    24K  none
lxd_pool/snapshots/teaching-crayfish                                                         24K  18.9G    24K  none

可以看到 layout 基本上是一樣的,只是全部的東西都裝在 mypool/lxd 而不是 lxd_pool,等於只是多一層而已

參考資料