necoro / feed2imap-go Goto Github PK
View Code? Open in Web Editor NEWA software to convert rss feeds into mails.
License: GNU General Public License v2.0
A software to convert rss feeds into mails.
License: GNU General Public License v2.0
Sometimes, feed2imap-go hangs without apparent reason. As this locks the cache file, all future runs just abort.
TIL: https://www.tigraine.at/2020/08/14/debugging-go-iowait-hang-sometimes-its-really-not-your-code -- send SIGABRT
to the running process to get a traceback.
This bug is to help my memory, when this occurs again.
Every two days or so I get a email from cron with
panic: send on closed channel
goroutine 213 [running]:
github.com/Necoro/feed2imap-go/internal/imap.(*Client).connect(0x400040a090, {{0x40001912b0, 0x5}, {0x40000c2738, 0x18}, {0x40001912d8, 0x8}, {0x4000191310, 0x10}, {0x5e228a, ...}, ...})
github.com/Necoro/feed2imap-go/internal/imap/client.go:72 +0x324
github.com/Necoro/feed2imap-go/internal/imap.Connect.func2(0x400034a540?)
github.com/Necoro/feed2imap-go/internal/imap/imap.go:46 +0x60
created by github.com/Necoro/feed2imap-go/internal/imap.Connect
github.com/Necoro/feed2imap-go/internal/imap/imap.go:45 +0x4ac
Not sure what to do about it, though. I guess the error could be caught and reported more prettily “E.g. saying which server failed during which operation”.
An error should probably be reported so that I know if it keeps failing.
However, a spurious error like this isn’t much to worry about; maybe feed2imap-go
could retry once or twice before bugging the user?
docker pull docker.pkg.github.com/necoro/feed2imap-go/feed2imap-go:latest Error response from daemon: Get https://docker.pkg.github.com/v2/necoro/feed2imap-go/feed2imap-go/manifests/latest: no basic auth credentials
Can you make this Docker Repo public or use docker hub?
https://ropenscilabs.github.io/r-docker-tutorial/04-Dockerhub.html
We won't support all configuration options from the old software. But, how are we dealing with them:
a) throw a warning
b) simply ignore
Currently preferred: a), as it will also spot typos
Currently, only IMAP support is planned; add Maildir support if needed / spare time at hand
The Arch RSS puts new items into the mailbox, even though they are so old they are automatically removed again.
In theory this should be handled by reupload-if-updated
, but perhaps this is broken. Investigate.
Example (from Dilbert): Can't Use Personal Laptop - Dilbert by Scott Adams
I have separate feeds of a newspaper that quite often share articles. This leads to having the same item thrice in the same folder.
Offer a way to merge items / have them only once.
As reported by @X41:
2021/10/17 03:34:33 ERROR: creating folder '': Invalid mailbox name: Name is empty (0.001 + 0.000 secs).
Up to now, feed2imap-go has (implicitly) required to use INBOX
or something alike as a top folder. But this was never the intention: Storing feeds directly under the root must also be possible.
(@X41: As I'm already working on it, I took the liberty of creating the issue myself :)).
Maybe clarify the documentation, or is this an enhancement request... I want all items to land in the same IMAP folder - I tried to leave out target (that creates a new folder for each source) and when I leave the target blank I get:
2020/08/15 05:00:33 INFO: Checking for folder 'Jobs/' 2020/08/15 05:00:33 ERROR: Creating folder of feed TNG: creating folder 'Jobs/': Mailbox already exists (0.001 + 0.000 secs).
...and it seems like no items are created.
The old ruby feed2imap used the original file names. It would be nice if this version did as well.
config.yml by design contains IMAP account credentials. Refuse to execute if it is world-readable.
In the release tarball, the example config is mode 0644.
Some feeds can return invalid certificates when talked to via HTTPS (wrong servername, self signed, missing cert chain). feed2imap used to circumvent this by always setting tls no verify
-- but now we want this to be done only on a per feed basis (plus the possible global option).
Until this is implemented, feeds with broken HTTPS are inaccessible
Currently, only the html rendering is implemented. The HTML-to-Text part is missing.
As per #25, it would be nice to have a new option auto-target
with the default yes
- which then can be set to no
and which will disable setting a target for each feed/group, thus sparing you all the null-targets
Checklist:
Using fastmail and the example config for XKCD, there are several issues.
The first I ran into was Message contains bare newlines
. The fix was something like:
re := regexp.MustCompile(`\s.*\r`)
contents := re.ReplaceAllString(msg.Contents, "\r\n")
Once that is resolved, I then receive the error Message contains invalid header
. See attached file for truncated log of the run.
log.txt
The username necorodm
is hardcoded in
As suggested on https://github.com/marketplace/actions/build-and-push-docker-images as well as in #15 (comment) I would propose not to hardcode the username and use a GitHub secret instead. This would make it easier for others to share their builds as well.
Is there an option to disable the grey boxed meta data?
Otherwise, great work. I was maintaining my own fork of "feeds2imap-go" for the past years but it stopped working and the IMAP functionality was very spotty. Is it related in any way? I added readability support for non-full feeds, which would be nice in your work too.
As stated in #66 (comment):
It scrapes a website for value content only and removes all the clutter. This can be used in some instances to turn a feed with preview content only into a full feed.
Instead of using the proposed https://github.com/go-shiori/go-readability, I created a trimmed down fork that avoids too much clutter in the dependencies.
I’m packaging feed2imap-go for nixpkgs at NixOS/nixpkgs#222690 and I am unsure what to do with the print-cache
binary. Is that something you’d expect to be included in a distro packaging?
Maybe a small README in https://github.com/Necoro/feed2imap-go/tree/master/tools/print-cache would be helpful.
The following options should be supported:
tls-no-verify
, cf. #3)The following options should be marked as unsupported:
-d
instead)Groups should allow options too. Those options will then apply for all feeds inside the group.
When changing from feed2imap to ~-go, one needs to build up the cache once in the beginning. For this, an option has to be created.
Currently, the number of connections is hardcoded to be 5 internal/imap/client.go#L16
Make this configurable, especially when concurrency is not wanted.
Thanks for providing a maintained version of feed2imap
, I am a log-time user!
Trying to migrate I run into this panic:
~ $ feed2imap-go -f ~/.feed2imaprc
2023/03/23 10:52:14 WARN: Unknown global option 'include-images'. Ignored!
panic: runtime error: index out of range [0] with length 0
goroutine 6 [running]:
github.com/Necoro/feed2imap-go/pkg/config.(*Url).RootPath(0xc000836a68?)
github.com/Necoro/feed2imap-go/pkg/config/url.go:177 +0x79
github.com/Necoro/feed2imap-go/internal/imap.Connect({{0xc00003e480, 0x5}, {0xc00002a9a8, 0x18}, {0xc00003e4a3, 0x8}, {0xc00003e4ac, 0x10}, {0xa4d362, 0x3}, ...})
github.com/Necoro/feed2imap-go/internal/imap/imap.go:33 +0x2dd
main.run.func1()
github.com/Necoro/feed2imap-go/main.go:137 +0x98
created by main.run
github.com/Necoro/feed2imap-go/main.go:135 +0x430
Maybe due to my old-style
target: imaps://YYYYY:[email protected]/XFeeds/Meine%20Bilder
entry?
Currently, in feed2imap, the connection string is given as
imap://account:pwd@host:port/Folder
.
This is ugly, especially as one has to take care to encode special chars (e.g. '@' accordingly).
--> Allow an alternative structure, f.ex.
connect:
protocol: imap # optional
host: foo.example
port: 993
user: account@foo
In feed2imap, target:
marked a feed to go into the top-level folder. Due to the way the Go-yaml-parser works, target:
cannot be distinguished from a missing target. Atm, one has to use target: ""
.
Goal: Make target:
work again.
The old feed2imap ordered the cached items ... but is this really necessary?
target
is a field that is defined globally, per feed, and per group.
In my world they were used to define the folder structure.
Turns out: In feed2imap, it was always possible to specify full URIs there, thus actually sending different feeds to different mailboxes.
I'm not sure if this is still needed -- normally you'd probably have multiple configs then.
As shown below, in feed2imap it was not uncommon to repeat the server target for each feed. This shall be supported. Different servers will remain unsupported.
Hiya - this project looks good - coming from feed2imap (the old one)!
I am getting some odd errors in this feed:
https://stackoverflow.com/jobs/feed?location=stockholm&range=50&distanceUnits=Km&allowsremote=True
The error looks like something like:
Error applying item filter: cannot fetch Name from *gofeed.Person (1:8)
| Author.Name != "Weirdo"
| .......^
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
go: github.com/Necoro/[email protected]: invalid version: unknown revision d9090592eb44
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.