tamarasaurus / immo-feed Goto Github PK
View Code? Open in Web Editor NEWA extensible app for scraping property listings
License: MIT License
A extensible app for scraping property listings
License: MIT License
To the UI
To the backend
Fix the accessibility
Describe the bug
When we remove results pages are never updated (it will be empty if we remove everything)
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Pages to be filled
Desktop (please complete the following information):
Additional context
Add any other context about the problem here.
E.g. the size helper needs to be totally rewritten and tested
At the moment every scraped result has the same date which makes the sorting useless
Instead of specifying a source url, implement a way to map the query parameters of each source to certain values
Move ports and volumes to docker-compose overrides
Update mongo service to use named volume
Extract latitude and longitude
Report which sites were scraped, how many results found
Add logging for storage, notifications
https://node-postgres.com/
https://github.com/salsita/node-pg-migrate
Add a way to paginate results by specifying the query param for page and looping until every result from the last day is stored
Also add pagination to the api results endpoint
In the backend - store multiple images, run migration
In the frontend - add number of images and click to preview modal for them
Add a component for the list of pages
Paginate the api in api/server
Paginate the mongo results
Only check the existence of the selector keys, not their values.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.