katharostech/docker-plugin_lizardfs

Docker volume plugin for mounting LizardFS

docker-plugin_lizardfs's Issues

share swarm volume

I got the following problem:
It seems, that docker swarm / the lizardfs plugin generates a volume name, with the stack name as a prefix.
With this behavior in general, how can I "share" a volume between two containers?
I think the path inside the lizardfs mount is the same as the volume name - so that they wont share data.

Example with two swarm stacks:

Stack: Documentation
Volume: DOC_DATA

Stack: CI
Volume: DOC_DATA

Both of them should point to the same amount of data, because my ci-server writes a bit of autogenerated code to the documentation part.

Am I able to achieve this behavior somehow?

Networking Broken on Docker 18.06

I just upgraded Docker on one of my Swarm hosts and I found that it seems Docker changed part of the swarm networking strategy so that the master does not recognized the chunkserver IP addresses correctly. There is some additional invisible container that joins to the swarm overlay network called lb-[network name] and the LizardFS master sees the chunkservers as joining to the master under that containers IP instead of the proper container IP.

I haven't done much research on this yet. For now I am just rolling back to Docker 18.03 and that fixed the issue. This problem should be fixed when I find time to refactor the plugin for Swarm.

LizardFS with docker plugin over VLAN (IPv6)

My current server structure, that gives me headaches, looks like this:

Data_Srv1 ("fd4a:1ffe:1cc:858a::11"):
- LizardFS Master (with docker-compose)
- LizardFS Chunkserver (with docker-compose)
- Doc Server (with docker swarm)

Mgmt_Srv1 ("fd4a:1ffe:1cc:858a::1"):
- CI Server (with docker swarm)

Data_Src1 and Mgmt_Srv1 are connected over a VLAN (and the docker swarm cluster), only accessable by IPv6 addresses. So far so good - the VLAN is okay, I tripple checked that.
The main problem I got is, that LizardFS only accepts IPv4 addresses - please correct me if I am wrong.

So to work around this problem, I created a ip4ip6 tunnel with socat, from my Mgmt to Data server:
socat TCP4-LISTEN:9421,fork TCP6:[fd4a:1ffe:1cc:858a::11]:9421

So, now comes the strange: The Doc-Stack (on Data_Src1) is able to create new files, fill them with content and delete them afterwards. But the CI-Stack (on Mgmt_Src1) is not - file creation is ok, but if I want to fill the file with content, the whole ssh session begins to hang.

I think it has something to do with the chunkservers not reachable from the Mgmt_Srv1? Are there any better approaches, especially for docker (which was the main reason, I created this ticket in this project 😁 )? (The LizardFS Containers do not have to be in swarm mode - I only want to have one Data-Server atm)

Build from image

Hi there

Thanks for all the work

Wondering if there is any info or documentation already on how to create the plugin from source

Thanks
Bruno

Rewrite in Rust

This is a tracking issue for rewriting the plugin in Rust. Not sure when/if we are going to get time todo it, but if we start using it more it is something we will want to do.

Lizardfs storage backend using snapshots

Not sure where to ask this, so apologies for posting it here - this seems to be the tip of the spear of knowledge about this - before I dive too deep into it, could this be a good start for a full storage backend for lizardfs, which uses lizard's snapshot functionality? Or would it have to be a complete rewrite?

plugin seems to be slowly filling up process table with <defunct>

Hi,

my VPSes are dying because there are not enough processes.
I traced it back to thousands of processes like that
root 31304 1149 0 12:30 ? 00:00:00 [mfsmount]
the father 1149 is
root 1149 1096 0 12:22 ? 00:00:01 node index.js

I am pretty sure it is the lizardfs plugin

I know too little to understand what's going on. Could you please help?
I could periodically reset my VPSes but it does not sound like the best option.

small files - Fuse3 client

Hi,

The plugin is awesome!
I only noticed that it's pretty slow with a lot of small files (e.g. sealife installation)
Would it be possible to replace fuse client with the new Fuse3 one?

Timeout tests are failing

Hi @zicklag. I am trying to build this for RPi and everything worked for me so far, till I tried to run tests.
All tests actually succeeded, apart of the last two, which are testing timeout for non existing lizardfs cluster.
Here is the end of my run-tests.sh:
[Plugin Test] Check timeout when connecting to non-existent cluster DRIVER VOLUME NAME lizardfs:latest lizardfs TEST FAILED [Root] All done. Stopping Docker cat: can't open '/run/dockerd-entrypoint.pid': No such file or directory sh: you need to specify whom to kill
I am not sure, why it behaves that way, but no matter what I did, docker volume ls always finish under 1s (usually time returns 0.56s). Maybe they changed the behavior in latest docker 18.06 (I am running 18.06.1-ce), I don't know.

The part with missing '/run/dockerd-entrypoint.pid' file is also interesting, as the file is actually visible from inside the running 'test-environment.sh' (where it is created), but apparently not from the docker-entrypoint.sh which tries to use it to kill that dockerd.

[Discussion] Plugin Networking Options

@zicklag is this still a problem with current docker swarm? I was trying to setup a lizardfs cluster like I have in https://github.com/onaci/docker-plugin-seaweedfs/blob/golang/seaweedfs.yml - fully in a private overlay network, used only for serving storage to the swarm via the volume plugin.

I expect that the work I'm doing in the seaweedfs volume plugin will be able to be applied to lizardfs just as easily as you did - so it would be really cool to compare them.

Originally posted by @SvenDowideit in #3 (comment)