RGW zongroup/zone placement 和 compression 用法

发表于 | 分类于 |

书接上文，上回书咱们了解了multisite的配置，最重要的是知道了zonegroup、zone是个什么东西，该怎么用。

那么今天我们来看看zonegroup和zone中的placement，顾名思义，用于告诉zone使用这个placement会将bucket、object要放到哪个里。

关于single rgw的模式，可以参考官网的使用配置，这里不做赘述了。下面介绍一下Multisite上的配置方法。

Placement

  1. 在master zone和非master zone上分别创建同名placement
  2. 使用的时候可以在用户信息中指定placement，也可以在创建bucket的s3请求中指定placement。

master zone

1
2
3
4
5
6
7
8
9
10
11
# radosgw-admin zonegroup placement add \
        --rgw-zonegroup default \
        --placement-id cold-placement

# radosgw-admin zone placement add \
        --rgw-zone exter \
        --placement-id cold-placement \
        --data-pool exter.rgw.cold.data \
        --index-pool exter.rgw.cold.index \
        --data-extra-pool exter.rgw.cold.non-ec \
        --compression zlib

看一下配置完的zone（此处忽略zonegroup配置完的接口）

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
# radosgw-admin zone get
{
    "id": "69234f1d-8f48-4431-b45f-912087431a47",
    "name": "exter",
    "domain_root": "exter.rgw.meta:root",
    "control_pool": "exter.rgw.control",
    "gc_pool": "exter.rgw.log:gc",
    "lc_pool": "exter.rgw.log:lc",
    "log_pool": "exter.rgw.log",
    "intent_log_pool": "exter.rgw.log:intent",
    "usage_log_pool": "exter.rgw.log:usage",
    "reshard_pool": "exter.rgw.log:reshard",
    "user_keys_pool": "exter.rgw.meta:users.keys",
    "user_email_pool": "exter.rgw.meta:users.email",
    "user_swift_pool": "exter.rgw.meta:users.swift",
    "user_uid_pool": "exter.rgw.meta:users.uid",
    "otp_pool": "exter.rgw.otp",
    "system_key": {
        "access_key": "85ZFW53VCKBZR7DQ7GS8",
        "secret_key": "MfyuJECq3Et2kktmf3T077rYwXQMEiIGYRlwKBnq"
    },
    "placement_pools": [
        {
            "key": "cold-placement",
            "val": {
                "index_pool": "exter.rgw.cold.index",
                "data_pool": "exter.rgw.cold.data",
                "data_extra_pool": "exter.rgw.cold.non-ec",
                "index_type": 0,
                "compression": "zlib"
            }
        },
        {
            "key": "default-placement",
            "val": {
                "index_pool": "exter.rgw.buckets.index",
                "data_pool": "exter.rgw.buckets.data",
                "data_extra_pool": "exter.rgw.buckets.non-ec",
                "index_type": 0,
                "compression": ""
            }
        }
    ],
    "metadata_heap": "",
    "realm_id": ""
}

让上述配置生效需要radosgw-admin period update --commit一下。

非master zone

1
2
3
4
5
6
7
8
9
10
11
# radosgw-admin zonegroup placement add \
        --rgw-zonegroup default \
        --placement-id cold-placement

# radosgw-admin zone placement add \
        --rgw-zone backup \
        --placement-id cold-placement \
        --data-pool backup.rgw.cold.data \
        --index-pool backup.rgw.cold.index \
        --data-extra-pool backup.rgw.cold.non-ec \
        --compression lz4

此处用的压缩算法与之前master zone使用的不一致，不影响使用。放心。rgw会在每个对象上记录一个压缩类型，所以即使你这一刻使用的压缩算法和之前使用的不一致也不影响解压工作。

看一下配置完的zone（此处忽略zonegroup配置完的接口）

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
# radosgw-admin zone get
{
    "id": "a6621518-6f80-41f5-a736-fb5d1814e036",
    "name": "backup",
    "domain_root": "backup.rgw.meta:root",
    "control_pool": "backup.rgw.control",
    "gc_pool": "backup.rgw.log:gc",
    "lc_pool": "backup.rgw.log:lc",
    "log_pool": "backup.rgw.log",
    "intent_log_pool": "backup.rgw.log:intent",
    "usage_log_pool": "backup.rgw.log:usage",
    "reshard_pool": "backup.rgw.log:reshard",
    "user_keys_pool": "backup.rgw.meta:users.keys",
    "user_email_pool": "backup.rgw.meta:users.email",
    "user_swift_pool": "backup.rgw.meta:users.swift",
    "user_uid_pool": "backup.rgw.meta:users.uid",
    "otp_pool": "backup.rgw.otp",
    "system_key": {
        "access_key": "85ZFW53VCKBZR7DQ7GS8",
        "secret_key": "MfyuJECq3Et2kktmf3T077rYwXQMEiIGYRlwKBnq"
    },
    "placement_pools": [
        {
            "key": "cold-placement",
            "val": {
                "index_pool": "backup.rgw.cold.index",
                "data_pool": "backup.rgw.cold.data",
                "data_extra_pool": "backkup.rgw.cold.non-ec",
                "index_type": 0,
                "compression": "lz4"
            }
        },
        {
            "key": "default-placement",
            "val": {
                "index_pool": "backup.rgw.buckets.index",
                "data_pool": "backup.rgw.buckets.data",
                "data_extra_pool": "backup.rgw.buckets.non-ec",
                "index_type": 0,
                "compression": ""
            }
        }
    ],
    "metadata_heap": "",
    "realm_id": "f334e6c2-c2b4-4c04-b541-f64d96c10c07"
}

同样为了上诉修改生效也需要执行一下radosgw-admin period update --commit

User中使用Placement

要想在User中配置Placement，首先得创建一个User。由于我们是在Multisite模式中创建的User，所以创建的动作必须在master zone上操作，如果在非master zone上操作用户不能同步到master zone上。具体原因请见Deploy multisite in 同一个集群

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
# radosgw-admin user create --uid colder --display-name colder

# radosgw-admin user info --uid colder
{
    "user_id": "colder",
    "display_name": "colder",
    "email": "",
    "suspended": 0,
    "max_buckets": 1000,
    "auid": 0,
    "subusers": [],
    "keys": [
        {
            "user": "colder",
            "access_key": "WGCQOD46GA92J7CIGXIY",
            "secret_key": "8yZw0FkYojiLdebWEF61HHTGhUvBrCel6ZzvGQRu"
        }
    ],
    "swift_keys": [],
    "caps": [],
    "op_mask": "read, write, delete",
    "default_placement": "",
    "placement_tags": [],
    "bucket_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "user_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "temp_url_keys": [],
    "type": "rgw",
    "mfa_ids": []
}

其中default_placement为空，此时会使用默认的placement也就是default-placement。所以需要配置这个选项。但是radosgw-admin中没有直接配置这个项目的参数，所以需要采用导出导入metadata的方式来修改该项目。

1
# radosgw-admin metadata get user:colder > colder.json

将user colder的配置导出到coder.json中，然后编辑其中的"default_placement": "cold-placement",，然后再将colder.json导入回去。

1
# radosgw-admin metadata put user:colder < colder.json

接下来可以使用该用户的access-key和secret-key去创建bucket并上传对象了。