Comments (9)
Yes, the design is flexible enough, so it is not an issue.
I'd like to avoid someone proclaiming "Hey, I am a Global Cache" when in fact they are keeping copies there a copy of their own data only :)
I would also like to avoid the endless discussion we had on the 24h in the WIS1 context.
The more we have SHALL (not only on this) the less we have uncontrolled moving parts in our design.
So, Global Cache SHALL keep a copy of all core data with cache: true
(when properties.cache
is missing, this is equivalent to true
).
Global Cache MAY keep a copy of cache: false
if they wish to do so.
@kaiwirt agreed ?
from wis2-guide.
If each GC can also decide not to cache data according to its own criteria, we should introduce a monitoring metric that captures the number so that sync comparison between GCs still remains possible.
Should the GC re-publish no message in such a case? Because if they simply distribute the message with the original data download URL this can lead to problems (both with the download on the recipient side and for the origin WIS2 Node).
Perhaps a report-notification could be sent instead of the data-notification. But in that case it has to be decided how the topic structure below reports should be, so that each centre can quickly filter relevant notifications for itself
from wis2-guide.
Global Caches shouldn't have the option not to cache data using their own criteria.
For those old enough to remember the endless debate between the GISCs and beyond on the option to select what is cached or not, I think, it is something we don't want to reopen in the WIS2 context.
So, by default, core data are cached. Except if the data producer decides (typically too large dataset, APIs,...) that its data shouldn't be cached.
Now, to address the issue raised here, that is where the GISCs comes into play. The idea would be to inform the GISC and the WIS2 Node via a notification message that "something" is wrong. It is likely that this "something" will require human intervention. So, GISC and WIS2 Node upon receiving the notification should act on it. Where and how those notifications are published would be the next step to discuss.
from wis2-guide.
@golfvert The main difference i see between caches in WIS1 and Caches in WIS2 is, that in WIS2 this is transparent. A user just follows the link. If that data is on Cache1 or Cache2 or on the producers endpoint is transparent.
So there is no need to have the same data everywhere.
I agree, that it is wanted that all caches have all core data. But you list a few exceptions (data size etc) which is not only a decision of the data producer. If a large data set can be downloaded is a decision of the cache.
So to cut a long story short: Caches should have all core data, but the design of WIS2 allows for more flexibility and there is no problem if a certain data set is missing at one cache (in contrast to WIS1).
from wis2-guide.
OK
from wis2-guide.
We still need to figure out the "meta-problem" of how an alert or issue about data is sent back to the originating centre (and/or it's affiliated GISC). This could be a "report" type message.
from wis2-guide.
This is about daily operations in WIS2 … so we’re looking for ET-OM to take a lead on proposing a solution.
from wis2-guide.
Agreed. This should be addressed by TT-OM.
from wis2-guide.
+1. Note that this was discussed at TT-WISMD this week. This will be required for GDCs to report back failures (KPIs will happen along the metrics/monitoring workflow).
from wis2-guide.
Related Issues (20)
- Add content for section on ODIS within interoperability with external systems HOT 2
- Create section describing interoperability between WIS2 and WHOS
- Develop non-regulatory material for WIS2 operations HOT 1
- Image captions don't render in HTML output
- In the Data Consumer part... HOT 7
- "+" signs don't always render in the documentation HOT 1
- Additional Guidance for data publishers HOT 2
- Need to add crossref to WCMP2 KPIs HOT 5
- Add section on WCMP2 KPIs
- Change "Manual on GDPFS" to "Manual on WIPPS" HOT 1
- Add definitions to Guide HOT 4
- Add guidance on how to make data_id unique
- Ensure all relevant metrics are included in the Guide
- Defining the mime type for WMO formats (GRIB and BUFR) HOT 4
- global service HTTP request identification HOT 2
- fix typo in part 2 HOT 1
- Global Cache web analytics HOT 1
- clarify whether to validate age of WNM pubtime
- add clarification on data updates
- add Global Replay Service HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from wis2-guide.