Comments (17)
Yes indeed this looks to be a big topic and when you are not aware and just face the issue it can be really frustrating. So some feedback about our experience through this issue.
The first point about the limit 500, each lambda produces min 4 resources:
- AWS::Lambda::Function
- AWS::Logs::LogGroup
- AWS::AppSync::DataSource
- AWS::AppSync::Resolver
If you consider a pattern like CRUD that is like 16 resources per Type. In addition you tend to also link your types together with sub-resolver, provide feeds for realtime (most of the time to bypass the AppSync limitation you tend to do a none datasource called internally) you can really quickly reach the 500. Like us we have a relatively small project and boom 526, from one day to another, your deployment is stuck.
Split stacks plugin, I got a look at it and it feels to be more like a workaround than a permanent solution. Even the author would consider more optimal to handle it manually than using the plugin.
Now come, nested stacks vs standalone/root stacks. Actually, there is also a limitation that has been open here, hard limit of 2.5k resources updated in per CF update. Okey the limit start to be big enough to have already something.
What about the performance? When you update with nested stack you have to update everything and it takes ages. We made the choice of multi services and 1 AppSync because it greatly follows the split of concerns/responsibility and we are able to quickly deploy only the part that has changed and by the members/team that own it. I fairly not see a way to correctly split a GQL schema in 2+ APIs (how would you sub-resolve something if not in this API? + pretty annoying to swap endpoints, I just dont see it possible)
The last point about ApiGateway (havent used serverless for that we did manually in past) but in terms of CF this is the same, you have an ApiId that need to be linked to methods/resources and then you can create routes the same way AppSync does. In addition, in the past, not using serverless you can still cross stack import Outputs to another stack that how we dealt with it.
If we want to go forward, standalone stack mean you can fine tune deployment permission using service role that you cant in nested since it use the root one.
.
You are also speaking about miss use of apiId? Do you think we can detect the valid usage and throw an error for the other usages? Maybe the question is does apiId can be use if you are not using service composition? Lets detect a compose file and warn/error the user about it?
Maybe also a documentation about how properly splitting your API could help. Pros, cons, what best suite multi developers same API? Personally I faced the issue of the 500 I could not find great documentation/pattern, I had to search in issues and make my own experiments and choice.
Last thing, this is pretty hidden piece of doc, but we discovered that serverless support TS config file. So since we needed to migrate from v1 to v2 and explode everything we just refactor in TS, made some helper functions to declare and split the resources and now it works just fine with serverless-compose.
We are also using offline mode (with the patch/PR for v2) because this helps a lot when you are several members working on the API (where deploy in the same env would override each other, different envs extra costs...) In the end the FE have local version why not BE? Oh and huge benefit you can run in isolation full locally the entire of your application ecosystem with a bunch of integration testing that gives you high confidence. So that is pretty awesome works that helps a lot, and big thanks for those plugins.
from serverless-appsync-plugin.
Hey @bboure, thanks for replying quickly.
We have the first service, in charge of deploying the API and the schema.
export const appSyncMain = {
name: `api`,
authenticationType: 'API_KEY',
schema: '../appsync/dist/*.graphql',
apiKeys: [
{
name: 'api-key',
description: 'basic api key',
expiresAfter: '30d',
},
],
mappingTemplatesLocation: '.',
defaultMappingTemplates: {
request: false,
response: false,
},
mappingTemplates: [],
dataSources: [],
functionConfigurations: [],
};
Then we have all the other services, in charge of plugging resolvers and mapping templates to that API and schema.
export const appSyncService = ({
apiId, dataSources, mappingTemplates, functionConfigurations,
}: AppSyncService) => ({
apiId,
schema: '../appsync/dist/*.graphql', // <---- HERE we have to pass the path of the schema again.
mappingTemplatesLocation: '.',
defaultMappingTemplates: {
request: false,
response: false,
},
mappingTemplates,
dataSources,
functionConfigurations,
});
The plugin requires specifying the path of the graphql schema. But that API / Schema has been deployed previously by the main service.
We end up with multiple updates on the schema. The AWS Core team told us this was provoking our issues with resolvers that got detached from the API.
We would like to know, how we can achieve to just do a reference to the APPSync API and Schema instead of modifying the resource.
Since we provide the apiId
, does it make sense to require the schema? Do we need to give the resourceId of the schema instead?
from serverless-appsync-plugin.
Hey @jeremycare I have the same use case as you. I've opened a PR and hope it will be merged soon. For now, Im using my fork on dev, but so far, everything works fine. I also considered your point about schema and agree that it should be managed from one place.
from serverless-appsync-plugin.
Hi,
What I'm not sure is why the service stacks update the schema?
Does each individual stack re-pass the (same?) GraphQL schema in the AppSync config?
from serverless-appsync-plugin.
Each stack is updating the same GraphQL schema, essentially making each stack a complete 'app' that uses the plugin once. This occurs a total of 15 times - once to create the API and the others to use the API ID to add resolvers.
Yes, they pass the same GraphQL schema between the stacks.
from serverless-appsync-plugin.
Okay seems that the v2 doesn't support apiId
anymore.
So I believe the plugin doesn't support our use case of AppSync right @bboure?
from serverless-appsync-plugin.
Should we fork and add our support ourselves or do you want us to do a PR to add this functionality?
from serverless-appsync-plugin.
Here's the architecture we have today and would like to have support with.
Also considering creating a custom plugin on our project that will correct the cloudformation stacks on top of this one to add a link instead of an update.
from serverless-appsync-plugin.
Hi there, I am looking forward to this because our team just reached the limit of resources per stack and so we have to split our mono service into multiple one keeping a single AppSync API.
Is there any progress on this? Workaround? or even post-install patches?
from serverless-appsync-plugin.
The workaround we are doing today is to do a depends_on in serverless-compose so we don't have concurrent updates (not ideal this the deployment time is very long now since we cannot do concurrent deployment on our microservices).
from serverless-appsync-plugin.
@Hideman85, I think it's a common topic on this repository, so you might find better solutions for your need in previous issues
from serverless-appsync-plugin.
@bboure Any thoughts on that?
from serverless-appsync-plugin.
This is a common debate/topic 🙂
I purposefully removed support for passing an apiId
in v2. In the past, it has been misused and would generate other issues. The problems you are facing are a symptom of that.
Serverless Framework (or any IaC) is not really made for deploying the same service into multiple stacks. An AppSync API should be considered as one service and it should not be broken apart.
When you look at how SF works "natively", there is no way to split an API Gateway into several stacks through compose
(as far as I know).
Right now, the only valid use case I maybe see is when reaching the stack limit. In general, 500 should be plenty enough, but I understand that in some cases, it might be reached. The workaround/preferred way of solving that issue for now is using https://www.npmjs.com/package/serverless-plugin-split-stacks
Eventually, I think what most of you are looking for is API federation/API merging.
Unfortunately, this is not available today.
I'm open to discussing this further and would be happy to try and provide solutions for all, but at the same time I'd like avoid introducing pitfalls.
from serverless-appsync-plugin.
Thanks for all the feedback.
I will take it into consideration and start thinking/planning about what is the best way to handle this.
from serverless-appsync-plugin.
After some research, I found this in the sls doc.
It looks like I was wrong when I said this is not supported for API Gateway.
I will go through this to see how we can "port" this to AppSync
from serverless-appsync-plugin.
It's important that the schema is not updatable when the APPSync API is referenced and not created by the plugin.
We had to talk to AppSync service team to troubleshoot our issues, and if there are multiple schema updates at the same time, cloud formation doesn't handle it correctly and some resolvers might disappear. (It caused us a lot of time to troubleshoot and find the issue for that kind of error, CF was not showing errors correctly :) )
from serverless-appsync-plugin.
Any update on that? @bboure
from serverless-appsync-plugin.
Related Issues (20)
- serverless-plugin-split-stacks: AppSync Api not found in stack. Did you forget to deploy? HOT 10
- TypeError: Cannot read properties of undefined (reading 'schema') HOT 1
- Validation does not match types ( request and response in resolverConfig )
- AppSync Arn reference error HOT 1
- UPDATE_FAILED: GraphQlSchema Property Validation Failure
- Subscription Type Error HOT 1
- Feature Request: Merged Api Support HOT 3
- Can't separate appsync into different file HOT 2
- v2.3.0 Update Drops/Recreates API HOT 3
- TypeError: Cannot read properties of undefined (reading 'schema') HOT 2
- Is the latest version not compatible with previous configuration? HOT 1
- Update existing app sync using appId
- Undefined Error using resolvers directive HOT 2
- Javascript resolvers supported now by AWS as Unit resolvers but not by the plugin HOT 5
- Upgrading from V1 > V2 drops + creates new API (breaks existing integrations) HOT 11
- The code contains one or more errors.. (Service: AWSAppSync; Status Code: 400; Error Code: BadRequestException, Proxy: null)
- appSync.logging.roleArn is not being detected when present
- get on the box,
- Possible vulnerability CVE-2021-4279 because of ajv-merge-patch / fast-json-patch HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from serverless-appsync-plugin.