Comments (5)
Very interesting, is there the possibility to refactor configuration and optparse (with argparse) and put also this feature? I think that there would be the Need to separate metadata Fetcher workers by mdx workers. In a production environment would be useful and more efficient to decide how many workers decide to run. Is It sustainable?
from pyff.
Yeah I think so although most people seem to favor generating the MDQ server as a batched set of static files these days and putting it all on a CDN for scalability. I like the idea tho!
from pyff.
Ok, good to hear this. I'd like to talk also about the followings.
- An SQL/nosql service where pyff's aggregator/validator classifies MD. It should work with an ORM and not with hardcoded queries. Mongodb, mysql, redis, PostgreSQL... in Django we can use all these with a normal and standardized syntax/approach/semantic.
- If pyffd MD workers could be batched with an autonomous scheduler like Cron or other not blocking approaches like celery/tornado/Twister all the outputs could be written to db
- MDX server could read from db result and answer to http request only
Sorry if I Say Django again but the 3th point Is veeery trivial with Django, nothing more to Say about URL regex patterns and security, It comes with batteries included! Even if db schema Will change Django schema migration sustem Will adapt the schema in semi-automatic way. If we could decouple mdx functions from http frontend I Will finalize a MDX in Django. Probably you already doing this, let me see the way, I Will follow this.
Also: I saw an mdx server in js in thiss project... Is this the final way?
from pyff.
from pyff.
from pyff.
Related Issues (20)
- Problem checking metadata signature with XRD method HOT 1
- XRD import doesnt work with x509 certificates that are expired HOT 5
- Request for IDPSSO still has SPs in resulting file HOT 2
- XML schema validation failure when signing HOT 6
- Filtering out entites HOT 4
- No parser for metadata signing certificates HOT 2
- MDQ response for non-standard request by identifier HOT 4
- Unexpected handling of xi:include element HOT 1
- distorted logo HOT 3
- mousless organization selection HOT 2
- Stylesheets and fork HOT 3
- Metadata of deleted XRD links are still published HOT 1
- Search HOT 9
- Filtering doesn't work as expected HOT 1
- Multi-word query strings don't match properly against metadata elements
- How do I add support for SHA1 in WSGI scenario HOT 1
- Python 3.10 - ImportError: cannot import name 'MutableMapping' from 'collections' HOT 1
- When filtering by entity type, I get a response that includes all entities HOT 11
- Pyff tries to write downloaded metadata to /var/run/pyff HOT 6
- Wrong entityId selected
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyff.