Comments (15)
Looks like anything less than 64K (!!) throws up the issue... which makes me think it's a problem my end.
I'll look in to it some more...
from next-http.
A few questions that can help me debug:
- Are you running on hardware or an emulator (if so which)
- Do you know your ESP versions - and can you paste here (to get them, on the spectrum run:
.uart
thenAT+GMR
) - Can you run the following and paste the result you see (from your computer rather than the speccy):
curl -i -X GET 192.168.0.6:8000/zxdb-loader.bas
I'll do some local tests too to see if I can replicate.
from next-http.
I've 'fixed' the issue locally - I just send extra data from my local server after the file itself, the make sure to flush everything...
<?php
// Save a router.php and run with:
// php -S 0.0.0.0:8000 router.php
@ini_set('zlib.output_compression', 0);
@ini_set('implicit_flush', 1);
$filename = $_SERVER["SCRIPT_FILENAME"];
header("Content-Type: application/octet-stream");
header("Content-Length: " . filesize($filename));
$fh = fopen($filename, 'r');
fpassthru($fh);
fclose($fh);
// Feed the client 64k of garbage to flush it's buffers (maybe???)
for ($i=0; $i < 65536; $i++) {
echo " ";
}
for ($i = 0; $i < ob_get_level(); $i++) { ob_end_flush(); }
ob_implicit_flush(1);
flush();
return true;
This isn't a 'proper' fix, as I'm sending more data than I should need to. I need to test a bit more with remote files as well to see how that fares.
I'll debug the issue a bit more over the next few days, and will send over the info you've requested.
Thanks again.
from next-http.
P.S. The file arrives on the Next with correct size, as it correctly used the Content-Length header - the extra bytes are just discarded. This isn't ideal, but I want to send stuff locally directly to the Next, so this'll work for now. I will get more info across to you though...
from next-http.
If I have time I'll fire up a local php server and run some tests too. The way .http
works is that it closes the connection once it has all the bytes it expected, so I can imagine you might have to flush, but I wouldn't expect it to need more bytes to sort ofβ¦ push the data over the wire π€
Still, glad you've got a work around π
from next-http.
Yeah, I'm not happy with my 'solution' (re: hack). The 64K size thing bugs me as well. Also, I tested with a Python local server - just not to be biased against PHP. The issue was the same - any file below 64Kb causes the issue on the Next - feels like a local networking issue to me - probably nothing to do with you in all honesty!
Anyway, here's the info you requested:
- Running on real hardware - this: https://ultimatemister.com/product/zx-next-official-clone-board/
- AT v1.1.0.0 (May 11 2016), SDK v1.5.4 (Compile time May 20 2016)
- See below:
β― curl -i -X GET 192.168.0.6:8000/zxdb-loader.bas
HTTP/1.1 200 OK
Host: 192.168.0.6:8000
Date: Wed, 19 May 2021 20:14:34 +0000
Connection: close
X-Powered-By: PHP/7.1.33
Content-Type: application/octet-stream
Content-Length: 188
Old firmware maybe? Scratch this - as the ZXDB downloader software works fine. Pretty sure it's something to do with my home set up. Going to try .http on some remote URLs with small file sizes in a bit.
from next-http.
The curl and versions look fine - older firmware, somehow, actually behaved better (we found that 1.6.0.0 introduced a pretty serious bug which I had to code around in the end!)
There's a capture-esp.bas
file in this repo that requests 4K from one of my servers and then captures a bunch of debug information at the same time. This might be useful as it gives me eyes into the ESP exchange. If you want to give that a go and send me the resulting 4k-esp-bank.bin
(or email it to me at [email protected]) then I can take a look to see the detail.
from next-http.
Thanks Remy. I'll get back on this in the next few days and provide you with the info you've requested.
What's weirder - I've done a copy $filename to #2
to dump the downloaded file to the screen, and part of the UART data has been saved to the start of the file. The corruption is always the same, +IPD,1460:
prepended to the start of the file (I've checked a couple of files now - they might even be corrupted all the way through - I need to check).
In this case, the file is a .scr file. I'll email over the original + received file at some point (will do a diff myself as well).
from next-http.
Please find attached a zip containing the .scr from the server, the received version on the Next, and 4k-esp-bank.bin.
from next-http.
Ah, it looks like I broke my own debugging tool! The bank dump was empty - it should have had useful info in it π€¦.
I'll get my debugging tool fixed (oh, you didn't modify the capture-esp.bas did you? it uses a debug build called http-debug.dot
in the same directory - I forgot to mention that).
If the +IPD
is in your file, this tells me that it's dropping data. The debug bank will help me confirm, but I suspect that's the case, but what's very strange is that it's not dropping for larger filesβ¦
Can I assume you're running the latest build of .http
?
from next-http.
I grabbed the latest code from here before running anything.capture-esp.bas
failed for me first of all as I didn't have the debug dot command in place, I made sure it was in place before running it the second time. I didn't modify the bas file in anyway (I had a look at the code to see which line failed then realised it used the debug dot file, which I dropped in place).
I've just tried with a different ESP module, and the result is the same (which also has the same firmware version it seems).
However, when using a hostname + port for the same file on a remote server - the file downloads perfectly (as you'd expect!)
Have you been able to reproduce the issue on your local network? I'm still wondering if it's an issue with http itself or something to do with my network setup (networking - always such fun!)
I'm happy to carry on looking in to this. I'd like to get it working locally as it would be an ideal way of pushing code to my Next - just drop the files on a local web server and grab it at the other end (without fiddling with SD cards etc). Not so ideal if I need to upload the files to the web first to download them!
Anyway, thanks for taking the time to look in to this. It's much appreciated. Let me know if there's anything else I can do to help.
from next-http.
What you're doing is all correct - looks like I broke my own test build! I'll fix that up and send it back to you for insights.
What I believe is happening under the hood is the the ESP is flushing messages too quickly for the machine to keep up, and so it overshoots the packet marker (the +IPD
thing). Which then means it slurps up the packet marker into your file and then when it's trying to keep reading, the socket has closed - which leads to the read timeout
.
What I don't understand (yet) is why having a larger buffer helps (though now thinking, it might be it allows the code to get enough data, but it's still missed packets and it leaves you with a corrupted file).
from next-http.
I've just double checked, and yes, larger files (>= 64Kb) do contain the IPD packet markers, so nothing is transferring reliably locally for me - larger files appear to work, but always have the +IPD,1460:
prepended to them.
from next-http.
Just to let you know I'm still on this - I need to fix up my debug build so it can get us the right information.
from next-http.
No worries - I appreciate it.
from next-http.
Related Issues (5)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from next-http.