Comments (18)
Alright,
how about trying the lean approach?
I will implement the feature, release the feature in alpha as experimental, you will test it out and if everything is OK, I can include it as an experimental feature in the standard release (and add tests for it). After some adoption/testing time, we can continue to push it as standard configuration property in one of the upcoming major/minor releases.
What do you think?
We are happy to go with this approach and test it out.
from gatsby-packages.
We have upgraded to V4 (4.0.0-alpha-9689ff.5) and it hasn't caused any noticable issues. Seems to have sped up page queries which is nice.
from gatsby-packages.
Out of interest: Could you share the numbers for the time boost?
This is only from a local single clean (no cache) build for each version so not exactly scientific but a definite improvement.
V3
success run page queries - 1470.570s - 11685/11685 7.95/s
V4
success run queries in workers - 908.137s - 11685/11685 12.87/s
from gatsby-packages.
I have released version
7.3.0-alpha.0
- could you test it out?I have described options in the Available options section - so for you, the value could be
system.collection=<COLLECTION_CODENAME>
.
We have tested this using a single collection as the filter and also mutliple (system.collection[in]=austria,default) and it works perfectly. Not only has it reduced the time to source and transform nodes but it has also reduced the time to run the page queries. I assume because the page queries now have less nodes to sort through.
from gatsby-packages.
@Simply007 I tested 7.3.0-alpha.0
by filtering our archived items
...
experimental: {
additionalItemFilterParams: `system.workflow_step[neq]=archived`,
}
...
Works as expected, thanks so much! I can't think of any filters outside of the system properties we would need to filter.
- Cross collection Links and Cross Links that aims to filtered-out item – if these are being filtered out to create individual sites, it makes sense to treat them as external links and train editors to handle them that way
- Webhook handling – this won't impact our project, but agree it would be nice to have additional options within Kontent for filtering webhooks based on criteria. We currently filter webhooks in the app that receives them as well
from gatsby-packages.
@Simply007,
I have been using gatsby-source-kontent/v/8.0.0-beta.1 and gatsby-kontent-components/v/8.0.0-beta.1 for over a week now and all seems fine so far. No issues to report.
from gatsby-packages.
@Simply007 Thanks! I have been using the filtering option in v8 and have no issues to report with this feature. Works great 👍
from gatsby-packages.
Curious if adding support for query params would be a good way to approach this. On our sites we would benefit from filtering content items by workflow steps (such as archive
— https://preview-deliver.kontent.ai/{{KENTICO_PROJECT_ID}}/items-feed?language=en&system.workflow_step=archived
) to limit the content sourced for builds. Collections can also be filtered this way according to the API. A more flexible solution that relies on the API's robust filtering method could enable a variety of capabilities.
One wrinkle to this approach is that it appears filtering by language is handled with a query param, so perhaps it would need to be documented as an optional additionalFilterParams
or similar to avoid complicating the implementation.
from gatsby-packages.
Hello @anthonyshughes and @bwlng,
thank you for reaching out!
I was thinking about the robustness of the solution. And @bwlng's idea about extending the configuration is pretty easy to implement but the robustness would not be really great - I will try to come up with something more complex, like a hook, but we can fall back to the easy string-based configuration for the queries. - Can you think of any other filter you might want to use? IMO it makes sense for filtering based on system
properties, right?
But in general, I have identified some possible problematic areas I would like to discuss prior to the implementation.
Cross collection Links
- How about linking an item from a different collection - Collection1 -> Collection2 using linked items element or rich text element. This item would not be in the model is that OK?
Cross Links that aims to filtered-out item
- How about linking an item from another workflow (any s property) using linked items element or rich text element. This item that does not fit the filter would not be in the model is that OK?
Webhook handling
- Currently there is no way of configuring webhooks to be fired only for specific collection items - are you OK that the processing would just skip these (maybe with some info log fr verbose output)?
I also have one more general question. I am testing gatsby v4 compatibility (#192), which is currently in alpha. What do you think about upgrading to the gatsby v4 with your project - my preliminary testing looks like there might not be any breaking changes from the kontent gatsby source plugin point of view.
from gatsby-packages.
Yes I agree that it would make sense to create a more generic solution to allow filtering on all the system properties rather than just collection.
Cross collection Links
From our point of view, we accept that linking to a collection that is filtered would cause an issue with links. Most of our editors only have access to the collections that are relevant to their site so this shouldn't be a problem for us. We would make it part of the editor training that they can't link to items from other collections for the couple of editors that do have access to multiple sites. I can see it could cause confusion for other customers if they are not aware of this restriction.
Webhook handling
Filtering webhooks by collection is a feature that I would like to see implemented within Kontent. We currently use an AWS Lambda function to receive the webhooks from Kontent and then redistribute them to our Gatsby Cloud sites depending on the collection the webhooks are for. This stops us building sites that do not have any changes. So not a problem for us specifically but again could be an issue for other customers.
I think that the potential issues that could be caused by implementing this feature could also be a problem for anyone using the Delivery Rest API. If these points are documented via the API filtering section then it shouldn't cause too many problems.
Gatsby V4
We will try upgrading and see what happens.
from gatsby-packages.
Alright,
how about trying the lean approach?
I will implement the feature, release the feature in alpha as experimental, you will test it out and if everything is OK, I can include it as an experimental feature in the standard release (and add tests for it). After some adoption/testing time, we can continue to push it as standard configuration property in one of the upcoming major/minor releases.
What do you think?
from gatsby-packages.
We have upgraded to V4 (4.0.0-alpha-9689ff.5) and it hasn't caused any noticable issues. Seems to have sped up page queries which is nice.
Great to hear! I still need to release the kontent gatsby source plugin to ensure that the source plugin is correctly using Gatsby v4.
Out of interest: Could you share the numbers for the time boost?
from gatsby-packages.
@bwlng @anthonyshughes
Alright,
how about trying the lean approach?
I will implement the feature, release the feature in alpha as experimental, you will test it out and if everything is OK, I can include it as an experimental feature in the standard release (and add tests for it). After some adoption/testing time, we can continue to push it as standard configuration property in one of the upcoming major/minor releases.
What do you think?We are happy to go with this approach and test it out.
I have released version 7.3.0-alpha.0
- could you test it out?
I have described options in the Available options section - so for you, the value could be system.collection=<COLLECTION_CODENAME>
.
from gatsby-packages.
Perfect @anthonyshughes!
I will keep this version out like that for a while so that you can send your feedback if you have some.
Then I will probably release a new version of the source plugin targeting gatsby v4 (#192) once it is out and stable.
Finally, I will include this feature as experimental in the release/or maybe include it as standard into the release if I have time to come up with a more robust solution.
from gatsby-packages.
Thank you both for the feedback @bwlng @anthonyshughes!
May I ask you for one more favor? Could you change the collection/workflow step to the one that is filtered in and vice versa?
I was wondering how does it behaves if you have the update webhook configured time-vise as well as for behavior itself. I guess it should remove/add item, but better to double-check.
from gatsby-packages.
Hey @bwlng and @anthonyshughes,
I have released v8.0.0-beta.0 with gatsby v4 support + filtering model feature:
- https://www.npmjs.com/package/@kentico/gatsby-source-kontent/v/8.0.0-beta.0
- https://www.npmjs.com/package/@kentico/gatsby-kontent-components/v/8.0.0-beta.0
Feel free to test it - feedback is welcome!
from gatsby-packages.
Version 8.0.0-beta.1 of packages:
- https://www.npmjs.com/package/@kentico/gatsby-source-kontent/v/8.0.0-beta.1
- https://www.npmjs.com/package/@kentico/gatsby-kontent-components/v/8.0.0-beta.1
Contains only deprecation fixes - functionality should not be affected.
from gatsby-packages.
I am closing the issue. Feature is released in https://github.com/Kentico/kontent-gatsby-packages/releases/tag/8.0.0
from gatsby-packages.
Related Issues (20)
- Gatsby Kontent Components - resolveLink in RichTextElement not getting called HOT 6
- ResolveDomNode for resolving links does not work correctly HOT 2
- Kontent workflow transition generates a preview build. HOT 11
- Image Flickers when conditionally rendering an image using React Context state HOT 13
- Type Error when using TypeScript HOT 7
- Publish webhook body versus Unpublish webhook body. HOT 11
- Showcase Gatsby SSR and DSG HOT 5
- Unable to use Deferred Static Generation with the Gatsby Source Kontent Plugin HOT 4
- Extend RichText resolver by component flag HOT 23
- Content backups are out of date
- Use new Netlify Gatsby plugin for examples and development site
- Replace Lerna with alternative HOT 2
- Export getGatsbyImageData
- [Spike] Make kontent assets as individual nodes
- Support React 18 in gatsby-components
- Support for content sync from gatsby
- Some observations from profiling Intralox HOT 1
- Asset renditions are not available in the Gatsby data layer HOT 14
- Build and Netlify deployments failing HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gatsby-packages.