COSBench是Intel团队基于java开发，对云存储的测试工具，全称是Cloud object Storage Bench。COSBench也分控制台和发起请求的driver，且driver可以分布式部署。可以支持swift、s3、Openstack等接口

获取

从github上下载release版本（https://github.com/intel-cloud/cosbench）。解压后可以获取COSBenchUserGuide.pdf（只有英文版）和workload配置文件（./conf/*.xml）。

部署&启动

COSBench的运行依赖于java和curl这两个包，并且调用linux的nc来做数据分析，如果没有安装nc，请手动安装。
本人将COSBench部署到docker中，为的是方便以后运行，不要将有限的生命耗费到无限的折腾当中。

docker镜像使用的是centos:7.3.1611，具体dockerfile如下：

FROM centos:7.3.1611

RUN yum update -y
RUN yum install -y java-1.8.0-openjdk

RUN yum clean all

EXPOSE 19088 18088

ADD ./cosbench.tar.gz /root/
ADD ./entry.sh /root/

ENTRYPOINT ["/root/entry.sh"]

entry.sh是容器的启动脚本，脚本中主要用于启动COSBench服务，具体实现如下：

#!/bin/sh

./start-all.sh

while true; do
    sleep 10s
done

完成docker镜像的生成后，使用docker run -d -w /root/cosbench/ --rm -P s3test:performance命令运行容器。然后通过docker ps查看可访问的端口。

1
2
3

$ docker ps
CONTAINER ID        IMAGE                COMMAND             CREATED             STATUS              PORTS                                                NAMES
ed37db78178a        s3test:performance   "/root/entry.sh"    19 hours ago        Up 19 hours         0.0.0.0:32771->18088/tcp, 0.0.0.0:32770->19088/tcp   unruffled_mahavira

在浏览器中输入http://x.x.x.x:32770/controller/index.html访问COSBench首页。COSBench是以”master-slave”模式运行的，19088端口对应的是master端口，18088对应的是driver端口，任务的运行是有master下发给driver，然后由driver执行的（driver可配置多个）。

关于workload配置文件的撰写请见COSBenchUserGuide.pdf，此处给出一个测试S3接口的例子

<?xml version="1.0" encoding="UTF-8" ?>
<workload name="S3Test" description="Performance">
    <storage type="s3" config="accesskey=Ltiakby8pAAbHMjpUr3L;secretkey=qMTe5ibLW49iFDEHNKqspdnJ8pwaawA9GYrBXUYc;endpoint=http://los-cn-north-2.lecloudapis.com" /> 
    <workflow>
    
        <workstage name="init">
            <work type="init" workers="1" config="cprefix=testp;containers=r(1,4)" /> 
        </workstage>
        
        <workstage name="prepare">
            <work type="prepare" workers="1" config="cprefix=testp;containers=r(1,2);objects=r(1,800);sizes=c(4)MB" />
        </workstage>
        
        <workstage name="sequential_RO_8W_4MB">
            <work name="sequential_RO_8W_4MB" workers="8" totalOps="1600">
                <operation type="read" ratio="100" config="cprefix=testp;containers=c(1);objects=s(1,800);size=c(4)MB" /> 
            </work>
        </workstage>
        
        <workstage name="random_RO_8W_4MB">
            <work name="random_RO_8W_4MB" workers="8" totalOps="1600">
                <operation type="read" ratio="100" config="cprefix=testp;containers=c(2);objects=u(1,800);size=c(4)MB" /> 
            </work>
        </workstage>
        
        <workstage name="sequential_WO_8W_4MB">
            <work name="sequential_WO_8W_4MB" workers="8" totalOps="1600">
                <operation type="write" ratio="100" config="cprefix=testp;containers=c(3);objects=s(1,800);sizes=c(4)MB" /> 
            </work>
        </workstage>
        
        <workstage name="random_WO_8W_4MB">
            <work name="random_WO_8W_4MB" workers="8" totalOps="1600">
                <operation type="write" ratio="100" config="cprefix=testp;containers=c(4);objects=u(1,800);sizes=c(4)MB" /> 
            </work>
        </workstage>
        
        <workstage name="cleanup">
            <work type="cleanup" workers="1" config="cprefix=testp;containers=r(1,4);objects=r(1,800)" />
        </workstage>
        
        <workstage name="dispose">
            <work type="dispose" workers="1" config="cprefix=testp;containers=r(1,4)" />
        </workstage>
        
        <workstage name="init">
            <work type="init" workers="1" config="cprefix=testp;containers=r(1,4)" /> 
        </workstage>
        
        <workstage name="prepare">
            <work type="prepare" workers="1" config="cprefix=testp;containers=r(1,2);objects=r(1,800);sizes=c(64)KB" />
        </workstage>
        
        <workstage name="sequential_RO_32W_64KB">
            <work name="sequential_RO_32W_64KB" workers="32" totalOps="1600">
                <operation type="read" ratio="100" config="cprefix=testp;containers=c(1);objects=s(1,800);size=c(64)KB" /> 
            </work>
        </workstage>
        
        <workstage name="random_RO_32W_64KB">
            <work name="random_RO_32W_64KB" workers="32" totalOps="1600">
                <operation type="read" ratio="100" config="cprefix=testp;containers=c(2);objects=u(1,800);size=c(64)KB" /> 
            </work>
        </workstage>
        
        <workstage name="sequential_WO_32w_64KB">
            <work name="sequential_WO_32w_64KB" workers="32" totalOps="1600">
                <operation type="write" ratio="100" config="cprefix=testp;containers=c(3);objects=s(1,800);sizes=c(64)KB" /> 
            </work>
        </workstage>
        
        <workstage name="random_WO_32W_64KB">
            <work name="random_WO_32W_64KB" workers="32" totalOps="1600">
                <operation type="write" ratio="100" config="cprefix=testp;containers=c(4);objects=u(1,800);sizes=c(64)KB" /> 
            </work>
        </workstage>
        
        <workstage name="cleanup">
            <work type="cleanup" workers="1" config="cprefix=testp;containers=r(1,4);objects=r(1,800)" />
        </workstage>
        
        <workstage name="dispose">
            <work type="dispose" workers="1" config="cprefix=testp;containers=r(1,4)" />
        </workstage>
    
    </workflow>
</workload>

问题及解决

COSBench首页时间显示没有按东八区显示
在cosbench-start.sh中找到/usr/bin/nohup java这一行，增加-Duser.timezone=Asia/Shanghai参数

COSBench在测试过程中MD5校验失败问题
在测试过程中GET操作失败，而且错误日志如下：

2017-07-11 07:26:42,297 [INFO] [Log4jLogManager] - will append log to file /root/cosbench/log/mission/MA308B012AE.log
2017-07-11 07:26:42,574 [INFO] [NoneStorage] - performing GET at /testp2/myobjects38
2017-07-11 07:26:42,575 [INFO] [NoneStorage] - performing GET at /testp2/myobjects43
2017-07-11 07:26:42,575 [INFO] [NoneStorage] - performing GET at /testp1/myobjects43
2017-07-11 07:26:42,575 [INFO] [NoneStorage] - performing GET at /testp2/myobjects31
2017-07-11 07:26:42,575 [INFO] [NoneStorage] - performing GET at /testp1/myobjects7
2017-07-11 07:26:42,575 [INFO] [NoneStorage] - performing GET at /testp2/myobjects26
2017-07-11 07:26:42,574 [INFO] [NoneStorage] - performing GET at /testp1/myobjects34
2017-07-11 07:26:42,574 [INFO] [NoneStorage] - performing GET at /testp2/myobjects25
2017-07-11 07:26:42,574 [INFO] [NoneStorage] - performing GET at /testp2/myobjects45
2017-07-11 07:26:42,574 [INFO] [NoneStorage] - performing GET at /testp1/myobjects7
2017-07-11 07:26:42,574 [INFO] [NoneStorage] - performing GET at /testp2/myobjects43
2017-07-11 07:26:42,576 [INFO] [NoneStorage] - performing GET at /testp2/myobjects16
2017-07-11 07:26:42,576 [INFO] [NoneStorage] - performing GET at /testp2/myobjects23
2017-07-11 07:26:42,576 [INFO] [NoneStorage] - performing GET at /testp1/myobjects15
2017-07-11 07:26:42,576 [INFO] [NoneStorage] - performing GET at /testp1/myobjects36
2017-07-11 07:26:42,576 [INFO] [NoneStorage] - performing GET at /testp2/myobjects34
2017-07-11 07:26:42,576 [INFO] [NoneStorage] - performing GET at /testp1/myobjects34
2017-07-11 07:26:42,576 [INFO] [NoneStorage] - performing GET at /testp2/myobjects27
2017-07-11 07:26:42,575 [INFO] [NoneStorage] - performing GET at /testp2/myobjects49
2017-07-11 07:26:42,575 [INFO] [NoneStorage] - performing GET at /testp1/myobjects27
2017-07-11 07:26:42,575 [INFO] [NoneStorage] - performing GET at /testp2/myobjects24
2017-07-11 07:26:42,575 [INFO] [NoneStorage] - performing GET at /testp1/myobjects35
2017-07-11 07:26:42,575 [INFO] [NoneStorage] - performing GET at /testp2/myobjects2
2017-07-11 07:26:42,575 [INFO] [NoneStorage] - performing GET at /testp2/myobjects47
2017-07-11 07:26:42,575 [INFO] [NoneStorage] - performing GET at /testp1/myobjects39
2017-07-11 07:26:42,575 [INFO] [NoneStorage] - performing GET at /testp1/myobjects50
2017-07-11 07:26:42,575 [INFO] [NoneStorage] - performing GET at /testp1/myobjects8
2017-07-11 07:26:42,575 [INFO] [NoneStorage] - performing GET at /testp2/myobjects21
2017-07-11 07:26:42,575 [INFO] [NoneStorage] - performing GET at /testp2/myobjects24
2017-07-11 07:26:42,575 [INFO] [NoneStorage] - performing GET at /testp2/myobjects15
2017-07-11 07:26:42,575 [INFO] [NoneStorage] - performing GET at /testp1/myobjects4
2017-07-11 07:26:42,575 [INFO] [NoneStorage] - performing GET at /testp2/myobjects1
2017-07-11 07:26:45,476 [ERROR] [AbstractOperator] - worker 8 fail to perform operation testp1/myobjects7
com.amazonaws.AmazonClientException: Unable to verify integrity of data download.  Client calculated content hash didn't match hash calculated by Amazon S3.  The data may be corrupt.
   at com.amazonaws.services.s3.internal.DigestValidationInputStream.validateMD5Digest(DigestValidationInputStream.java:79)
   at com.amazonaws.services.s3.internal.DigestValidationInputStream.read(DigestValidationInputStream.java:61)
   at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:72)
   at com.amazonaws.services.s3.model.S3ObjectInputStream.read(S3ObjectInputStream.java:155)
   at com.amazonaws.services.s3.model.S3ObjectInputStream.read(S3ObjectInputStream.java:147)
   at com.intel.cosbench.driver.operator.Reader.copyLarge(Reader.java:120)
   at com.intel.cosbench.driver.operator.Reader.doRead(Reader.java:92)
   at com.intel.cosbench.driver.operator.Reader.operate(Reader.java:69)
   at com.intel.cosbench.driver.operator.AbstractOperator.operate(AbstractOperator.java:76)
   at com.intel.cosbench.driver.agent.WorkAgent.performOperation(WorkAgent.java:197)
   at com.intel.cosbench.driver.agent.WorkAgent.doWork(WorkAgent.java:177)
   at com.intel.cosbench.driver.agent.WorkAgent.execute(WorkAgent.java:134)
   at com.intel.cosbench.driver.agent.AbstractAgent.call(AbstractAgent.java:44)
   at com.intel.cosbench.driver.agent.AbstractAgent.call(AbstractAgent.java:1)
   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   at java.lang.Thread.run(Thread.java:748)
2017-07-11 07:26:45,476 [INFO] [NoneStorage] - performing GET at /testp1/myobjects14
2017-07-11 07:26:46,880 [INFO] [NoneStorage] - performing GET at /testp2/myobjects14
2017-07-11 07:26:49,806 [INFO] [NoneStorage] - performing GET at /testp1/myobjects43
2017-07-11 07:26:51,514 [INFO] [NoneStorage] - performing GET at /testp2/myobjects31
2017-07-11 07:26:54,021 [INFO] [NoneStorage] - performing GET at /testp1/myobjects5

你可以通过增加-Dcom.amazonaws.services.s3.disableGetObjectMD5Validation=true参数来关闭MD5校验，据说此bug只在0.4.3.c4版本出现，我没验证过其它版本。

COSBench搭建及使用

获取

部署&启动

问题及解决

参考&鸣谢