blang / vfs Goto Github PK

Virtual filesystem library written in golang

License: MIT License

Go 100.00%

vfs's Introduction

vfs for golang

vfs is library to support virtual filesystems. It provides basic abstractions of filesystems and implementations, like OS accessing the file system of the underlying OS and memfs a full filesystem in-memory.

Usage

$ go get github.com/blang/vfs

Note: Always vendor your dependencies or fix on a specific version tag.

import github.com/blang/vfs

// Create a vfs accessing the filesystem of the underlying OS
var osfs vfs.Filesystem = vfs.OS()
osfs.Mkdir("/tmp", 0777)

// Make the filesystem read-only:
osfs = vfs.ReadOnly(osfs) // Simply wrap filesystems to change its behaviour

// os.O_CREATE will fail and return vfs.ErrReadOnly
// os.O_RDWR is supported but Write(..) on the file is disabled
f, _ := osfs.OpenFile("/tmp/example.txt", os.O_RDWR, 0)

// Return vfs.ErrReadOnly
_, err := f.Write([]byte("Write on readonly fs?"))
if err != nil {
    fmt.Errorf("Filesystem is read only!\n")
}

// Create a fully writable filesystem in memory
mfs := memfs.Create()
mfs.Mkdir("/root", 0777)

// Create a vfs supporting mounts
// The root fs is accessing the filesystem of the underlying OS
fs := mountfs.Create(osfs)

// Mount a memfs inside /memfs
// /memfs may not exist
fs.Mount(mfs, "/memfs")

// This will create /testdir inside the memfs
fs.Mkdir("/memfs/testdir", 0777)

// This would create /tmp/testdir inside your OS fs
// But the rootfs `osfs` is read-only
fs.Mkdir("/tmp/testdir", 0777)

Check detailed examples below. Also check the GoDocs.

Why should I use this lib?

Only Stdlib
(Nearly) Fully tested (Coverage >90%)
Easy to create your own filesystem
Mock a full filesystem for testing (or use included memfs)
Compose/Wrap Filesystems ReadOnly(OS()) and write simple Wrappers
Many features, see GoDocs and examples below

Features and Examples

Current state: ALPHA

While the functionality is quite stable and heavily tested, interfaces are subject to change.

You need more/less abstraction? Let me know by creating a Issue, thank you.

Motivation

I simply couldn't find any lib supporting this wide range of variation and adaptability.

Contribution

Feel free to make a pull request. For bigger changes create a issue first to discuss about it.

License

See LICENSE file.

vfs's People

Contributors

Stargazers

Watchers

Forkers

gernest gitter-badger lotrfan dwlnetnl chiaen wuman jbuchbinder yhzhao ligadous leobcn adammck j1m1l0k0 lalinsky valery-barysok eugene-dounar etsangsplk mikalv startime-h dsharp-pivotal richardyicai wheelcomplex happy-ferret xakep666 wade-welles kimbsen doytsujin backwardn lu4p linuxerwang pepesi jmvoid arkadiy93 iam-justin ofshellohicy cyrainfish shpaky andrey1s luxingwen llhhbc selonsy dream-kzx wnz27 andrusgerman izouxv jmcconnell26 3job vforks karelbilek

vfs's Issues

memfs seems not work correctly?

Hi there,

I am interested to use this in my minimum docker image, basically the idea is that mount /tmp as tmpfs for the program to use. As it is minimum, there is no mount command, etc, just the app binary it self.

However from my test it seems not to use memory fs, that is after the docker container stop, I can see the file content is on the filesystem. So not sure if there is a problem with it, or the way I expect it to work is totally wrong.

Here is my snippet

package main

import (
	"fmt"
	"io/fs"
	"io/ioutil"

	"github.com/blang/vfs"
	"github.com/blang/vfs/memfs"
    u "github.com/sunshine69/golang-tools/utils"
)

func main() {
	fmt.Println("started")
	var osfs vfs.Filesystem = vfs.OS()
	err := osfs.Mkdir("/adir", 0777)
    u.CheckErr(err, "Error")
	mfs := memfs.Create()
	err = mfs.Mkdir("/tmp", 0777)
    u.CheckErr(err, "Error")
    ioutil.WriteFile("/tmp/test", []byte("hello my file"), fs.ModeAppend)
    //Seems not work, after run I can inspect the container and the file does exist. If it is in memory it should not exist
}

Dockerfile

FROM stevekieu/golang-script:20210804 AS BUILD_BASE
RUN mkdir /app && mkdir /imagetmp && chmod 1777 /imagetmp

ADD . /app/
WORKDIR /app
ENV CGO_ENABLED=0 PATH=/usr/local/go/bin:/opt/go/bin:/usr/bin:/usr/sbin:/bin:/sbin

ARG APP_VERSION
RUN go mod download golang.org/x/net
RUN go build -trimpath -ldflags="-X main.version=v1.0 -extldflags=-static -w -s" --tags "osusergo,netgo,sqlite_stat4,sqlite_foreign_keys,sqlite_json" -o minimum-docker
CMD ["/app/minimum-docker"]

FROM scratch
# the ca files is from my current ubuntu 20 /etc/ssl/certs/ca-certificates.crt - it should provide all current root certs
ADD ca-certificates.crt /etc/ssl/certs/
COPY --from=BUILD_BASE /app/minimum-docker /minimum-docker
COPY --from=BUILD_BASE /imagetmp /tmp
ENV TZ=Australia/Brisbane
ENTRYPOINT [ "/minimum-docker" ]

Build the image, and run. After run use docker inspect the container name to find the UpperDir location, and get in there, the /tmp exists a file with the content. If it is memfs, then it should not exists any file at all.

Thanks

Filesystem.OpenFile returns vfs.File, not *vfs.File

I was wondering why OpenFile doesn't return a pointer to a File, like its standard library counterpart. I would say that mirroring the standard library as closely as possible makes it a lot less painful to migrate to using vfs as a wrapper.

Exec?

This is probably very silly, but I'd like to be able to run one of the files in the virtual filesystem. Is this something that's supported, or is this a terrible idea that I should immediately forget about?

S3 implementation

Hello @blang

Maybe you know by any chance the s3 implementation of your Filesystem interface? If not would be interested in having one even if you claim "Only Stdlib" (this will bring aws sdk as a dependency)?

Thank you in advance.