suse/lrbd

lrbd

Simplifies iSCSI management of Ceph RBD images.

The lrbd utility centrally stores the configuration in Ceph objects and executes the necessary rbd and targetcli commands to apply the stored configuration.

The tool depends on targetcli which depends on python-rtslib. The original purpose is to support the rbd backstore and is the default option. If your distribution is lacking kernel support or updated packages for targetcli and python-rtslib, the iblock backstore can be used for many configurations.

Quickstart

For the impatient, follow the instructions for the simplest configuration of one initiator to one gateway to one image. A working Ceph installation is required.

From gateway

1. Install lrbd
   • On SUSE: # zypper in lrbd
   • On Fedora: # dnf install lrbd
   • On RHEL 7: # yum install lrbd
2. Create a pool
   # ceph osd pool create swimming 256 256

• Create an RBD image
  # rbd -p swimming create raft --size 2048
• Create an initial configuration
  # lrbd -e
• Replace archive with raft and rbd with swimming. Remove the initiator (and the comma from the previous line). Replace the host igw1 with the result of uname -n. Save the file. Run lrbd using an iblock backstore.
  # lrbd -I
• Inspect with targetcli
  # targetcli ls

From client

1. Discover the target.
   # iscsiadm -m discovery -t st -p gateway_address

• Login
  # iscsiadm -m node -p gateway_address --login
• Find device
  # multipath -ll
• Format, write data, read data
• Logout
  # iscsiadm -m node -p gateway_address --logout
• Delete discovery cache
  # iscsiadm -m node -o delete

From gateway

1. From the gateway, clear the configuration
   # lrbd -C

• Unmap the RBD images
  # lrbd -u
• Wipe the configuration from Ceph
  # lrbd -W

Manual

See Wiki

FAQ

• What is a gateway?

The gateway is simply a Linux host that supports iSCSI. This host is providing iSCSI access to another host, a client or initiator, and is mapping Ceph RBD images locally. Any client that cannot use Ceph directly (lack of library support), but supports iSCSI can still use Ceph storage.

One gateway can support multiple iSCSI targets which is primarily the reason for the distinction.

• Where should lrbd run?

For creating or editing the configuration, lrbd can run on any ceph client with proper credentials (e.g. ceph.client.admin.keyring). For applying the configuration, lrbd runs on the host intended to be a gateway.

• What about configuring multipath?

Lrbd does not configure multipath on an initiator. See multipathd.

lrbd's Issues

"Disable restart of service during future updates" is not in master

The "Disable restart of service during future updates" commit (7385458) is in the 1.0 branch, but it's not in master.

It's typical to have all changes go to the master branch first, and then cherry-pick the specific ones that you want to the stable release branches.

Would you mind cherry-picking that one to master?

lrbd missing RHEL 7

The lrbd package doesn't seem to be available on the latest RHEL 7... Is there another repo I should be using?

yum install lrbd

Loaded plugins: product-id, search-disabled-repos, subscription-manager
No package lrbd available.
Error: Nothing to do

[Question] Lun number & sync config overall iSCSI gateways

Hi team,

Actually, I'm looking for a storage solution for my system kubernetes.
I need authenticate and I have a farm ceph. Because i choose iSCSI and project Lrbd for this solution. I have some questions.

• When i run command "#lrbd -H gw". It automation map lun to image ceph, Have ever lun number changed? if I clear "#lrbd -C" and generate "#lrbd -H gw" again.
• I used command "#lrbd -f config.json". When I change my config.json, I must change config.json all gateways iSCSI then run "#lrbd -H gw". Do you have sync config overall gateways iSCSI.

Thanks team <3.

honor /etc/sysconfig/lrbd from command line

Custom options such as keyrings can already be added to LRBD_OPTIONS in /etc/sysconfig/lrbd. This works fine for systemd commands.

Running lrbd at the command line can be a little cumbersome having to specify the same options again. These should be pulled into the environment, but still allow command line options to override them. An info/warning should likely inform the user what the real/equivalent command being run is. That likely means we need an option to disable this as well.

The man page will need an update.

please tag releases

Would you mind tagging releases in Git? Just a simple annotated tag is fine, like 1.0.2 or 1.0.3.

No way to get LUN information

After adding a new TPG to export a new target, there is no way to collect the Lun IQN, name, and # from the new target with lrbd.

When I create a new TPG target with lrbd, I can't find out what the LUN name or LUN id is for the new target with lrbd.

use targetcli clearconfig instead of tcm_fabric --unloadall

tcm_fabric was part of the old lio-utils package, which is no longer shipped on many distros. With modern targetcli-fb , the clearconfig command should be used instead.

"make tests" fails on master

Some of the changes in commit f5407a7 caused make tests to no longer work properly.

No way to delete a target/LUN

I'm trying to use lrbd to write an openstack cinder driver that exports rbd images as Cinder volumes. I need to be able to create targets and remove them at will. There seems to be no way to remove an existing target with lrbd.

If I edit my config.json and remove an existing TPG entry, and then rerun lrbd, the LUN still exists in targetcli ls.

Consider using rtslib directly instead of via targetcli

Targetcli is a pretty face (command completion etc) on top of rtslib, which is an actual pythonic api to modify LIO's configuration. You may find that a considerable amount of output parsing and class wrappers in lrbd can be simplified by using rtslib directly.

You may wish to look at my targetd project, which uses an alternate rtslib than suse likely ships (rtslib-fb instead of rtslib) but its block.py is a similar example of combining lio configuration with something else, and using rtslib to minimize the extra code users must write to configure lio.

Need to be able to install lrbd from pypi

It seems the only way to install lrbd on a host is from RPM. since lrbd is a python app, I would like to be able to install it from the standard pip install command line. This enables me to isolate the python dependencies into it's own virtualenv and make the install safe.

I posted up a pull request for a refactoring of lrbd here:
#17

This allows one to push up lrbd into pypi as a standard python application.

iSCSI multipath, Can it possibly lead to data inconsistency?

Hello everybody,
I using lrbd deploy 2 gateways iscsi, 1 client iscsi connect to 2 gateways (multipath gateway A and gateway B, use alua active/enable).
I have a question, when i am writing data to iscsi A, A fail, data continue write to iscsi B. Can it possibly lead to data inconsistency?
Sorry i'm bad english.
Thanks.

Implement FirstBurstLength

/iscsi/iqn.20...143ed577/tpg1> set parameter FirstBurstLength=262144

"git describe" on master is older than on "1.0" branch

Would you mind tagging the tip of the master branch? Maybe you could tag it as "1.1.0" or something?

Currently "git describe master" prints "1.0.2", whereas "git describe 1.0" prints "1.0.4", which is confusing.