Comments (7)
Hi
Using awss3
as backend, on create a new file and submit to s3 bucket, when we finish the sync/close the cachefile is removed with success, but the directory still exists ( with no files ).
It is not a big problem for me, but is there a way to remove the cachefile + path
For me, the workaround is: once I finish to process my files, I will remove all content of /tmp
( or what I define on the configuration ).
My project is a daemon, I will process files from one s3 bucket to another in my main loop, I don't want to create hundreds of empty directories so I will force delete.
from cloudstorage.
Thanks for the report, will try to upgrade the tests to detect this and get patch in as soon as I can.
from cloudstorage.
@araddon I believe we'll want to remove the Release
method and have all the stores clean up in Close
... Originally calling Close didn't clean up the cache file on any of the stores, one had to Close and then call Release to free up resources. So, it seems like we have a mix of both right now?
from cloudstorage.
Actually it looks localfs is the only remaining store that does't cleanup in Close(). I'd say, lets fix it and drop the Release func from the interface.
from cloudstorage.
Going to fix this in a small PR and then open up a new Issue to cleanup the release interface.
from cloudstorage.
@peczenyj - That second problem you mentioned seems worthy of it's own issue. Could you create a new issue for cleaning up cachedirs when they become empty? I also have comments about that, but I'll wait until you create a new issue.
from cloudstorage.
@peczenyj - That second problem you mentioned seems worthy of it's own issue. Could you create a new issue for cleaning up cachedirs when they become empty? I also have comments about that, but I'll wait until you create a new issue.
I rethink and seems the temporary directory is my responsibility. so i will be Explicit in create and clean it, and it will be fine.
of course will be great delete the files, and the empty dirs can be removed from time to time since it take less space.
thanks for your time!
from cloudstorage.
Related Issues (20)
- google storage type BucketLifecycleRuleCondition field Age change type to *int64 HOT 2
- AWS S3 write closer does not wait for the inner goroutine to close
- Add support for Azure Blob Storage
- Add support & Interface for Event Based File Listening Model
- embed os.FileInfo in object as interface HOT 1
- local-fs has different folder paths than the cloud (google, aws)
- localfs NewWriter, should use O_TRUNC ? HOT 3
- should be s3 not google HOT 1
- Overwriting behavioral difference between stores types, when using store.NewWriter... HOT 4
- improve sftp support
- sftp improvements
- box.com integration
- S3 Copy/Move performance optimizations HOT 1
- awss3 List method ignores NextMarker
- awss3 List does not call query ApplyFilters HOT 2
- Tests failing after I modified Move/Copy test cases to use variable len payloads
- Using the Google auth for JWT files w/ out a scope can lead to confusion.
- AWS S3 set endpoint to be used in yandex s3 HOT 3
- LocalFS Prefix Query Does Not Match Filename Prefixes
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cloudstorage.