Comments (5)
Hi Simon,
Thanks for sharing and thanks for the feedback! I did not know about httpie
but I immediately installed it. That looks like a real time saver. 😍
It's funny that I had also originally intended to use the problem with curl
& jq
in a bash
script and run it regularly with crontab
. My exact use case is to query for new Postings regularly and to be alerted when new Postings are matched. To be honest, I went with the Golang solution because I haven't used it in years and I always wanted to try it out on my raspberry pi. 😅
Anyways, I'm a big friend of bash
and you inspired me to quickly hack and share my initial idea.
https://github.com/RomanNess/fundgrube-crawler/blob/main/.sh/fundgrube-crawler.sh
- The script also iterates over
outlets
since the API does not supportoffset > 990
. - Pagination is not supported because I have never seen >100 postings in an Outlet.
- It saves discovered deals in a "DB file" per search term and should only discover "new Postings".
Run the thing in a crontab
and alert about discovered postings via mail and it does fulfil the requirements of my initial use case. 🙂
If you intend to tinker with the API yourself I have documented my findings in https://github.com/RomanNess/fundgrube-crawler/tree/786291daa899c46ad4e44cd117802663b69a69ef#api-peculiarities.
from fundgrube-crawler.
https://github.com/simonneutert/fundgrube 🛒
cron running 🥳
from fundgrube-crawler.
lets add a "json differ" and pipe forward to https://github.com/caronc/apprise 🤯
from fundgrube-crawler.
I was thinking of good-old Gnu MailUtils for alerting from bash. 😅
https://mailutils.org/manual/html_section/mail.html
from fundgrube-crawler.
marry me 🥳
closed for public 🚧 http://t.me/simonneutert
from fundgrube-crawler.
Related Issues (1)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fundgrube-crawler.