ncarlier / feedpushr Goto Github PK

View Code? Open in Web Editor NEW

324.0 7.0 24.0 45.61 MB

A simple feed aggregator daemon with sugar on top.

License: GNU General Public License v3.0

Makefile 1.14% Go 75.97% Dockerfile 0.24% HTML 1.71% JavaScript 1.21% CSS 0.10% Shell 0.31% TypeScript 19.32%

rss rss-aggregator feed-aggregator-daemon go restful-api push twitter mastodon opml

feedpushr's People

Contributors

Stargazers

Watchers

feedpushr's Issues

OIDC client_id and secret parameters

I'm trying to setup OpenID auth (via Google) for feedpushr, but I noticed that the only parameter requested for setup is FP_AUTHN (for the issuer URL), whereas OIDC additionally requires a client id and secret (and the redirect URI I guess) to complete the flow.

Is there a way to pass these to feedpushr on start up like FP_AUTHN, or is the recommended pattern to clone the repo and make the modifications locally? If the later, can you point me to where the secret is set in the code + the redirect URI format? I've found that the client_id is set to 'feedpushr-ui' so that I can change.

Thank you for creating feedpushr btw, it's a great app that has saved me tons of time - I appreciate the work you've put into it!

[Feature request] masonry style dashboard with articles

Hi again,

Hope you are doing well !

Would be awesome to have some masonry backed dashboard to display the aggregated feeds' articles.

Cheers,
X

chore(install) install feedpushr using Ansible

Hi,

I made a 1st version of an Ansible to install Feedpushr: https://github.com/slundi/ansible_feedpushr

It uses latest release binary to install Feedpushr. But plugins are only available for AMD64 arch because they are not on the ARM and ARM64 archive.

Is it OK to add the Ansible install doc on the README.md?

not aggregating feeds despite being activated

Hi again,

Hope you are doing well !

I doesn't seem to be aggregating my feeds, counters remain at zero.

Nothing happened after the 54s.

Any action that I missed ?

Feeds:

Cheers,
X

Empty on restart

Running a feedpushr in a docker container, when the container is restarted all of the data disappears.

I assume the solution is to set up a volume, however the folders to use in the volume are not listed in the repo.

Could someone please let me know which folders I need to link.

Thanks

unsupported content type: content-type

Hi,

Hope you are all well !

When I want to add this feed, https://morioh.com/feed, it triggers an alert box saying "unsupported content type: content-type".

Any explanation to why it happens ? as it sounds a proper rss feed.

Cheers,
Luc

Is there a variable for feeds `Title` to be used in the output?

I've successfully sent feed outputs to Mattermost instance as written in the document but is there a variable to send feed's Title (not the article title but the title put in the admin panel for each feeds)?

I have set multiple feeds to be sent to a single Mattermost output configuration but I can distinguish each messages better if I can see which feed it came from.

OPML import does not support categories

Here is a sample OPML :

<?xml version="1.0" encoding="UTF-8"?>
<opml version="1.0">
    <head>
        <title>No matter what</title>
    </head>
    <body>
        <outline text="Info" title="Info">
            <outline type="rss" text="LinuxFr.org : les dépêches" title="LinuxFr.org : les dépêches" xmlUrl="http://linuxfr.org/news.atom" htmlUrl="https://linuxfr.org/news"/>
        </outline>
    </body>
</opml>

Nothing is imported and the following error appears in the logs :

WRN unable to create feed: skipped error="Get : unsupported protocol scheme """ component=import title="No matter what" url=

Enhancement : Add a button to force fetch for a particular feed

Feeds are enabled on restart

I imported feeds using OPML.

Once feeds are imported, they are all disabled, which is fine.

If I stop then restart feedpushr, all feeds are enabled :

I think they shall remain disabled since I did not enable them.

Proxy support?

Is there any proxy support, specifically on a per feed basis? Basically to get around Cloudflare...

How to use RDBMS plugin through docker?

I first tried to place the plugin in the environment variable, (not sure if the path was right and I tried several others like feedpushr-rdbms.so, contrib/rdbms/feedpushr-rdbms.so,

working_dir: /var/opt/feedpushr
environment:
  - FP_PLUGINS=plugins/feedpushr-rdbms.so

but those paths failed with realpath failed in the log.

I also tried to place feedpushr-rdbms.so from the downloaded release file of the same version of the latest docker image at 3.2.0 but this failed with plugin was built with a different version of package internal/abi.

Is there a way to add an output plugin to the docker instance?

plugin rake is not appearing

Hi again,

Hope you are all well !

In order to build the yake module, I wanted to test the rake plugin.

./feedpushr --log-pretty --plugin feedpushr-rake.so

Nowhere, where to configure the rake parameters.

Am I missing something ?

Cheers,
X

How to make condition work in case insensitive?

In other words, how to make matches and include in EXPR engine work in case insensitive?

Add tag even without pressing 'Enter' key if tag input field is dirty

If the tag text field is dirty (I'm adding a tag) nothing is added when I click "Update".

I think most of the users will add a single tag, thus won't press enter before clicking the "Update" button.

[Output plugin request] : Apprise

Thanks for the software.

Is it possible to add Apprise as output plugin?

https://github.com/caronc/apprise

Plugins

I dont know how to get plugins for docker to work. If i add this to my docker compose the container wont start

environment:
  - FP_PLUGINS=`./plugins/feedpushr-twiter.so,./plugins/feedpushr-rake.so`

Installation
Copy the feedpushr-rake.so file into your Feedpushr working directory.

I can not fine a file called feedpushr-rake.so

How to load the database file from host for Docker installation?

I'm trying to run this via docker compose but I'm not sure how I can achieve this to bind mount the database file (and the .htpasswd file) from the host. Otherwise all the data gets wiped after down and up.

[Feature request] throttling the number of output per second

Hi again,

Hope you are still all well !

I have developed a couple of plugins that require to throttle the output.

For example, I have written a plugin twitter-selenium. It launch one chrome instance but it needs to wait that one tweet is written/sent before having another tweet to be triggered.

Also, this apply for the yake module, if too much requests are sent to the service, it makes the rest service unavailable quickly because we do not have an option to throttle the output.

This feature is really important as it gonna be required for many potential plugins.

Thanks again if you can push that feature.

Cheers,
Luc

[Feature Request] add yake

Hi,

Hope you are all well !

It would be awesome to create a contrib for yake.
https://github.com/LIAAD/yake

The idea would be to create a docker-compose and use the rest api of yake.
https://github.com/LIAAD/yake#option-2-rest-api-server-in-a-docker-container

I will fork the repo and try to implement it asap.

Any advice or insights to share upfront ?

Cheers,
X

[Feature Request] add pagination to feed list

Hi,

I imported 1200 feeds into feedpushr and can see only 100.

I would be awesome to add pagination to browser other feeds.

Here is the opml file for example:
https://github.com/x0rzkov/feedpushr/blob/master/resources/security.opml

Cheers,
X

[Bug] concurrent map iteration and write error

Hi,

Hope you are all well !

I have this error message when exiting feedpusr (ctrl/c).

Any way to fix it ?

Cheers,
Luc

[Feature request] add htmlUrl in the feed list

Hi,

Hope you are doing well !

It would be awesome to display a small icon with the link to the the blog, when available, near the feed link in the ui.

It would make easier the choice to enable/disable a feed in feedpushr.

Thanks again for this great tool !

Cheers,
Luc

env var is incorrect in README

The env var APP_STORE is listed in the docs as the datasource uri for bolt. The correct var is APP_DB.

[Feature request] Lack of access UI auth

Hey! I'm really liking Feedpushr for RSS. It's done nicely, it's fast and scrapes a lot of content very accurately. But there's a lack of any auth - literally anyone can access my UI and play with my RSS feeds, get to know my API key etc.

I think it would be a good move to require some kind of auth, like login and password.

OPML imports in a strict mode

Hi, @ncarlier - thank you for all the good work.

I have a use case where I would like to have an OPML file define a portable source of truth for which URLs to follow.

Importing it via the API would make feedpushr obey the items included in the last import, discarding the rest, which differs from the append-only approach as it exists now.

I'm happy to provide an example of this. The POST endpoint could take a payload/param signalling this strict import mode.
Is there any appetite for it / pitfall to be aware of before I make a start?

cannot make plugins

Hi,

Hope you are all well and congrats for this new release :-) !

While building the plugins, I have the following error:

feedpushr % make plugins
>>> Building: feedpushr-twitter.so 7a823e0 for linux-amd64 ...
# github.com/ncarlier/feedpushr-contrib/twitter
loadinternal: cannot find runtime/cgo
/usr/local/Cellar/go/1.14/libexec/pkg/tool/darwin_amd64/link: running clang failed: exit status 1
ld: unknown option: -z
clang: error: linker command failed with exit code 1 (use -v to see invocation)

make[1]: *** [plugin] Error 2
make: *** [plugins] Error 2

Any reason why ? Is there a way to sort it out ?

Cheers,
X

Google authentication error

I trying:

feedpushr.exe -authn https://accounts.google.com

but Google don't like it.

Error 401: Client_id was not found

Repeated output of same articles

I've added this feed and it seems both http and rdbms output keep repeating the same entries over and over on every interval.

https://ubuntu.com/security/notices/atom.xml

Cannot add slow feed as DefaultTimeout is too small

// DefaultTimeout for HTTP requests
const DefaultTimeout = time.Duration(5 * time.Second)

I self-hosted RSS-Bridge on Celeron PC and using its component, XPathBridge which is resource demanding task.
The default 5-second timeout leads to context canceled.
May I ask you to increase the default timeout or add a settings for it?

Mail output plugin - no Date

I'm on 3.3.1 in Docker.

Using mail output plugin results in emails being without dates at all. No such header added here.

Some mail clients does not show any date for mails if no such header present. This not a problem for Thunderbird as it shows date on which email was received, but for KMail2, for example, a huge one:

Those "неизвестное" should be a date on which mail was sent.

Some sites parsed incorrectly

Hello, I'm in docker on e38a6ee with mail output plugin.

Some sites parsed incorrectly, e.g. sometimes new releases from github repositories appears like:

and no actual release information.

Confirmed feeds:

Github releases.
https://linux.org.ru site (like https://www.linux.org.ru/section-rss.jsp?section=1 feed).

It happens absolutely randomly, sometimes it parses feed normally, sometimes it puts something like HTML head in letter (like on screenshot).

I was using latest release before, it was working fine.

Can't add RSS feeds from Reddit

Hey, I'm on 3.3.1 in Docker.

Can't add RSS feeds from Reddit. In web interface it says "http error: 403 Forbidden", yet curl downloads feed (for example, https://www.reddit.com/r/selfhosted.rss) just fine, from the very same machine.

Not all articles are being sent to output plugin?

I've added an ATOM feed from https://status.digitalocean.com/history.atom and the feed screen is showing 25 as Aggregation success but in the output plugin that I've configured to send to Mattermost (I do not have any rate limiting in Mattermost) with http plugin, I only get the latest 2 article entries but should it not send the entire 25 entries to the output plugin once the feed and the output settings are added?

I've also added https://status.linode.com/history.atom and this one sent out 4 articles to the output plugin and another from https://www.cloudflarestatus.com/history.atom sent 9 articles to the output plugin and seems it's behaving inconsistently.

failed to compile ui

Hi,

Hope you are all well !

while running the command npm run start, I have the following error:

Failed to compile
/app/src/feeds/FeedList.tsx
TypeScript error in /app/src/feeds/FeedList.tsx(23,16):
Generic type 'Column<RowData>' requires 1 type argument(s).  TS2314

    21 | }
    22 | 
  > 23 | const columns: Column[] = [
       |                ^
    24 |   { 
    25 |     title: 'Aggregation',
    26 |     render: (feed: Feed) => ( !!feed && <FeedControl feed={feed} /> ),

Any way to fix it ? Thanks in advance.

Have a good week-end !

Cheers,
X

How to configure webhooks with Gotify ?

Target URL: https://push.xxx.com/message?token=XXXXXIEEO7hrhQ
Content Type: JSON
Payload:

{
  "title": "[feedpushr]",
  "message": "{{.Content}}",
  "priority": 8
}

It didn't work !

twitter plugin credentials

Hi,

Hope you are doing well !

How do you pass the twitter's credentials to the twitter plugin ?
Can you prodive an example of command line ?

Cheers,
X

{"level":"error","error":"unsuported filter: ","time":X,"message":"unable to init filter chain"}

[sevos]:<~>$ feedpushr -addr 127.0.0.1:9001 -filter fetch
{"level":"info","component":"store","uri":"boltdb://data.db","time":1580562047,"message":"using BoltDB datastore"}
{"level":"error","error":"unsuported filter: ","time":1580562047,"message":"unable to init filter chain"}
{"level":"fatal","error":"unsuported filter: ","time":1580562047,"message":"unable to init main service"}

I'd love to have full text, so I tried to run it with the fetch plugin, however it's not working for some reason. Any idea what's wrong?

OPML categories should be turned into tags

When importing OPML, the file may contain some categories ( cf #2 ).

It may be useful for users to keep these categories.

It can be done by turning these categories into tags

Is it possible to use the your service with third party clients ?

I think of the ios App "Fiery Feeds" .

Do i need feedpushr and reeder or can feedpushr installed seperatly ?

Do you have a demo instace of feedpushr ?

Ruby API Client

Just in case this is useful to anyone. I used the swagger.json doc to generate a ruby api client gem here - https://github.com/bluengreen/feedpushr_client

Thanks for the great project.

ncarlier / feedpushr Goto Github PK

feedpushr's People

Contributors

Stargazers

Watchers

Forkers

feedpushr's Issues

Recommend Projects

Recommend Topics

Recommend Org