目前我们有两种路径使用Ceph的块存储：?
- 利用QEMU/KVM通过librbd与 Ceph 块设备交互，主要为虚拟机提供块存储设备，如下图所示；?

- 利用kernel module与Host kernel交互，主要为物理机提供块设备支持。

Librbd 是Ceph提供的块存储接口的抽象，它提供C/C++、Python等多种接口。对于C++，最主要的两个类就是RBD?和?Image。?RBD?主要负责创建、删除、克隆映像等操作，而Image?类负责映像的读写等操作。

准备工作

对于任何客户端应用，都需要首先连接到一个运行良好的Ceph集群。

获取集群句柄

//声明Rados对象，并初始化 librados::Rados rados; ret = rados.init("admin"); // just use the client.admin keyring if (ret < 0) { // let's handle any error that might have come back std::cerr << "couldn't initialize rados! err " << ret << std::endl; ret = EXIT_FAILURE; return EXIT_FAILURE; } else { std::cout << "we just set up a rados cluster object" << std::endl; } //获取配置文件信息: /etc/ceph/ceph.conf // 1. 根据命令行参数 /* ret = rados.conf_parse_argv(argc, argv); if (ret < 0) { // This really can't happen, but we need to check to be a good citizen. std::cerr << "failed to parse config options! error " << ret << std::endl; ret = EXIT_FAILURE; return EXIT_FAILURE; } else { std::cout << "we just parsed our config options" << std::endl; // We also want to apply the config file if the user specified // one, and conf_parse_argv won't do that for us. for (int i = 0; i < argc; ++i) { if ((strcmp(argv[i], "-c") == 0) || (strcmp(argv[i], "--conf") == 0)) { ret = rados.conf_read_file(argv[i+1]); if (ret < 0) { // This could fail if the config file is malformed, but it'd be hard. std::cerr << "failed to parse config file " << argv[i+1] << "! error" << ret << std::endl; ret = EXIT_FAILURE; return EXIT_FAILURE; } break; } } } */ // 2. 程序里面指定 ret = rados.conf_read_file("/etc/ceph/ceph.conf"); if (ret < 0) { // This could fail if the config file is malformed, but it'd be hard. std::cerr << "failed to parse config file! err " << ret << std::endl; ret = EXIT_FAILURE; return EXIT_FAILURE; }

连接集群

ret = rados.connect(); if (ret < 0) { std::cerr << "couldn't connect to cluster! err " << ret << std::endl; ret = EXIT_FAILURE; return EXIT_FAILURE; } else { std::cout << "we just connected to the rados cluster" << std::endl; }

创建I/O上下文环境

如果没有存储池，需要先新建一个存储池。

新建存储池

const char *pool_name = "gnar"; ret = rados.pool_create(pool_name); if (ret < 0) { std::cerr << "couldn't create pool! error " << ret << std::endl; ret = EXIT_FAILURE; rados.shutdown(); //断开集群连接 return EXIT_FAILURE; } else { std::cout << "we just created a new pool named " << pool_name << std::endl; }

新建I/O上下文环境

librados::IoCtx io_ctx; //I/O上下文 const char *pool_name = "gnar"; ret = rados.ioctx_create(pool_name, io_ctx); if (ret < 0) { std::cerr << "couldn't setup ioctx! err " << ret << std::endl; ret = EXIT_FAILURE; rados.shutdown(); //断开集群连接 return EXIT_FAILURE; } else { std::cout << "we just created an ioctx for our pool" << std::endl; }

若干RBD映像API

声明RBD对象，创建rbd映像