mayth / go-simple-upload-server Goto Github PK
View Code? Open in Web Editor NEWSimple HTTP server to save artifacts
License: MIT License
Simple HTTP server to save artifacts
License: MIT License
I use this image for a little while.
It is great, but it seems to have nothing about modifying file name automatically.
"Automatically" means that I upload from browser, so I can't change the filename, before upload.
Could you add feature on this?
I upload a file. I got the message by postman.
{"ok":false,"error":"invalid URL escape "%\x1d(""}
Hi !
Im trying to use this api but the CORS Policy is disabled on the server.
Is there a way to enable it easily ?
Thanks !
Hi,
download a file fails when bandwith is poor / low
i can reproduce when limiting bw with curl
for example :
❯ curl --limit-rate 1200k https://needed.xxx.xxx/files/ffmpeg_6_1 -H "Authorization: Bearer XXX" -o /tmp/ffmpeg
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
34 75.2M 34 25.6M 0 0 1147k 0 0:01:07 0:00:22 0:00:45 967k
curl: (92) HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)
the error:
curl: (92) HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)
the file size to download is 75mb (not so large)
Regards
I start this server by command ./server tmp
. Then upload a file by below command:
curl [email protected] 'http://localhost:25478/upload?token=b30731d81df7fc22338d'
An error will return like this:
{"ok":false,"error":"open tmp/util.go: no such file or directory"}
I have this form:
<html>
<head>
<title>Test upload</title>
</head>
<body>
<h2>Select files to upload</h2>
<form name="upload" method="POST" enctype="multipart/form-data" action="https://cdn.masvino.tv/content/upload?token=dca5f3c43c44d4b97">
<input type="file" name="file1"><br>
<!--input type="file" name="file2"><br>
<input type="file" name="file3"><br-->
<input type="submit" name="submit" value="Upload">
</form>
</body>
</html>
After submiting I got this error:
{
ok: false,
error: "http: no such file"
}
Docker container logs:
time="2020-03-06T12:06:03Z" level=error msg="failed to acquire the uploaded content" error="http: no such file"
And Nginx logs:
root@h0579:/var/cache/nginx/client_temp# tail -f /usr/local/nginx/logs/error.log
2020/03/06 09:17:10 [warn] 12376#12376: *41 a client request body is buffered to a temporary file /var/cache/nginx/client_temp/0000000011, client: 168.194.206.109, server: cdn.masvino.tv, request: "POST /content/upload?token=3714ed2fd1dca5f3c43c44d4b972f34ff27242b3 HTTP/1.1", host: "cdn.masvino.tv"
2020/03/06 09:17:10 [warn] 12376#12376: *41 a client request body is buffered to a temporary file /var/cache/nginx/client_temp/0000000011, client: 168.194.206.109, server: cdn.masvino.tv, request: "POST /content/upload?token=3714ed2fd1dca5f3c43c44d4b972f34ff27242b3 HTTP/1.1", host: "cdn.masvino.tv"
2020/03/06 09:18:05 [info] 12376#12376: *40 client closed connection while waiting for request, client: 168.194.206.109, server: 0.0.0.0:443
I planned to use this project as a backup endpoint on my Raspberry Pi but noticed that OOM happens about halfway through an upload of 4.7 GB archive (I've build an image for Arm64v8).
The memory usage peak of this container was 3.8 GB before being killed due to OOM. My Rpi has 4GB RAM.
Due to my lack of knowledge of Go (I'm learning), I cannot say whether is this This seems to be by design (File stored in memory before flushing out to the disk) or there might be a memory leak, but the RAM usage pattern doesn't support that theory well.
EDIT: Found the issue (or at least one part of it): https://github.com/mayth/go-simple-upload-server/blob/master/server.go#L83
This function seems to load the whole file into the memory.
Would you be interested in baking only the resulting Go binary into a Alpine Image to reduce the Docker image size?
hi,
thanks for that project.
FYI:
I made a simple RPM with a corresponding systemd-Job.
See: https://github.com/FalcoSuessgott/go-upload-server-rpm
If you interested in a PR. I could submit one.
Could you add a simple HTTPS support please?
At the moment there is no upload to Subdirectories possible, uploaded files will be always stored in the root directory of /upload. If a curl with /dir/file is executed you will always get "curl: (22) The requested URL returned error: 404 "
See following examples:
curl -X PUT -k -f -Ffile=@ /files//?token=
or
curl -X POST -k -f -Ffile=@ /upload/?token=
I am running the server in a kubernetes statefulset.
Generally the upload is working. The problem is that the MaximumFileSize I set doesn´t seem to have any affect.
I also understand that this error should be Logged which it isn´t.
time="2021-04-08T07:26:13Z" level=info msg="starting up simple-upload-server" time="2021-04-08T07:26:13Z" level=info msg="start listening" cors=false ip=0.0.0.0 port=25478 root=/var/root token=<yes> upload_limit=72118430
The server returns this:
(413) Request Entity Too Large.
the file size is around 7500Kb so the upload size I set should be plenty enough. I have tried several diffrent upload sizes.
I don´t know much about go but as my upload size is logged correctly I figure it probably parses correctly as well.
Btw. it would be great to have support for docker exec :)
Does your image support chunked upload?
I'm seeing ./simple_upload_server.go:66:12: undefined: NewServer
when I try to use this.
Is there some module it needs?
Hey there,
i just found your project last week and used it in a test environment for integration-testing.
Unfortunately, your changes last weekend broke my integration as I was using the :latest tag.
It would be awesome if you could publish your release tags also as docker image tags on dockerhub.
Would really appreciate that.
Best regards!
go-simple-upload-server/server.go
Line 38 in 09aae11
Hello,
Thanks for your work, it's a pleasure to play around with.
I wanted to flag an issue with CORS support.
When the browser sends a POST
request to a server, it first sends an OPTIONS
request to verify that the servers returns a valid Access-Control-Allow-Origin
header.
(Issue reproduced from both Chrome and Firefox)
From Stack Overflow:
OPTIONS requests are what we call pre-flight requests in Cross-origin resource sharing (CORS).
They are necessary when you're making requests across different origins in specific situations.
This pre-flight request is made by some browsers as a safety measure to ensure that the request being done is trusted by the server. Meaning the server understands that the method, origin and headers being sent on the request are safe to act upon.
Your server should not ignore but handle these requests whenever you're attempting to do cross origin requests.
To fully support CORS, the post suggests the following:
A way to handle these to get comfortable is to ensure that for any path with OPTIONS method the server sends a response with this header:
Access-Control-Allow-Origin: *
I am currently trying to upload a html file to a container running your image, by using a "HTTP Request" node in n8n.
The previous step in my workflow writes to "/tmp/index.html", I have no way of providing a path without a directory so far.
Can I POST to your container using something like "@/tmp/index.html" also?
Additional: what about cleaning up? How to remove the file after processing things?
Thanks for any help, Stefan
When running with a config file as
$ go-simple-upload-server -config config.json
where config.json
contains
{
"addr": "127.0.0.1:8081"
}
I will get the following error (because 8080 is in use for something else on my system)
2024/01/07 12:20:07 Start listening on 127.0.0.1:8080
2024/01/07 12:20:07 server stopped: unable to listen on 127.0.0.1:8080: listen tcp 127.0.0.1:8080: bind: address already in use
But, if I intentionally set addr
to an empty string (so bypass triggering of the default flag value), it works:
$ go-simple-upload-server -addr "" -config config.json
2024/01/07 12:21:32 Start listening on 127.0.0.1:8081
2024/01/07 12:21:32 Start serving on 127.0.0.1:8081
Thinking this is related to the merging strategy used to combine flags and the config file, because if I remove the WithOverride
strategy the config file works:
diff --git a/app.go b/app.go
index afaf1e9..7b4b279 100644
--- a/app.go
+++ b/app.go
@@ -50,7 +50,7 @@ func main() {
if err := json.NewDecoder(f).Decode(&fileConfig); err != nil {
log.Fatalf("failed to load config: %v", err)
}
- if err := mergo.Merge(&fileConfig, config, mergo.WithOverride); err != nil {
+ if err := mergo.Merge(&fileConfig, config); err != nil {
log.Fatalf("failed to merge config: %v", err)
}
config = fileConfig
with the consequence that config file options will always take precedence over command line options, which is not entirely intuitive.
Is there one method to delete files?
Maybe use /files/ as prefix for path is too much.
Could you please remove /files/ prefix of path by default?
$ ./simple-upload-server -token foo ./tmp
$ curl [email protected] 'http://localhost:25478/upload?token=foo
{"ok":true,"path":"/files/tmp/test.txt"}
expected: "path":"/files/test.txt"
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.