http fails to execute after offline mutation

[awslabs]

http fails to execute after offline mutation

[danielvtan]

running the sample (no edits. just follow the instruction)
https://github.com/aws-samples/aws-mobile-appsync-events-starter-android

produces the same error.

steps:
turn off internet
create event
after creating event, turn internet on

[rohandubal]

Does this only happen on first offline mutation or all offline mutations? Do the normal mutations go through fine?

[danielvtan]

This happens to all of the offline mutation. Sometimes the first offline mutation go in.
Normal mutations are fine.

Take note. the aws appsync android sample also does not work. Same error.

I have a solution that is currently working but after about a hundred of call it fails.
too many open files error. or sometimes a syqlite error
i removed the client checking so now my app always creates a new appsync client.
doing this made it work not sure why. but eventually ffails so i need a real solution here.

//if(client == null){
            client = AWSAppSyncClient.builder()
                    .context(context)
                    .cognitoUserPoolsAuthProvider(getUserPoolAuthProvider(context))
                    .region(Constants.APPSYNC_REGION)
                    .serverUrl(Constants.APPSYNC_API_URL)
                    .persistentMutationsCallback(persistentMutationCallback)
                    .build();
//}
``

[danielvtan]

I tested a bit more.
Doing offline mutation closing the app and opening it up again then turning the internet makes it work.
So basically i think the persistent cache is working. The in memory is the one that fails.

I think that is also the reason why my temp solution works because it forces appsync to store the mutation in the persistent cache.

[rohandubal]

Thanks for the details @danielvtan I will investigate the issue.

[danielvtan]

I got a temporary fix working. I edited the appsync sdk a bit. Here is what i did.

I removed the in memory mutation by removing this line

//AppSyncOfflineMutationManager.java -> addMutationObjectInQueue
inMemoryOfflineMutationManager.addMutationObjectInQueue(mutationObject);

to replace the in memory mutation i always call this after a mutation.

//PersistentOfflineMutationManager
persistentOfflineMutationObjects = fetchPersistentMutationsList();
        persistentOfflineMutationObjectMap = new HashMap<>();
        for (PersistentOfflineMutationObject object: persistentOfflineMutationObjects) {
            persistentOfflineMutationObjectMap.put(object.recordIdentifier, object);
        }
        Log.d("AppSync", "There these many records in persistent cache: " + persistentOfflineMutationObjects.size());

the purpose of the code above is to ditch the in memory mutation entirely
after that here is the actual temp solution
in AppSyncCustomNetworkInvoker.java -> executeRequest i call this in the run function

OkHttpClient okHttpClient = (OkHttpClient) httpCallFactory;
okHttpClient.connectionPool().evictAll();

after doing this, offline and online mutation are finally working.
for the full explanation i am not really sure but i got the idea from this issue square/okhttp#3146 which is still open and not resolved

Please make an actual fix for this. thank you!

[donaldarmstrong]

I see the same exception... The mutation is longer available and our client and backend are out of sync.

D/AppSync: onFailure()Failed to execute http call
D/AppSync: Got message to take action regarding mutation queue.
D/AppSync: Got message that a originalMutation process errored out.
D/AppSync: Checking if I need to process next originalMutation
D/AppSync: First check: Internet Available
D/AppSync: Second check: Persistent mutations queue is EMPTY!
D/AppSync: Third check: Inmemory mutations queue is EMPTY!

2018-11-14 19:24:13.381 10200-10261/com.regiscorp.athena.pos E/TransactionSDKImpl: Failed to execute mutation com.apollographql.apollo.exception.ApolloNetworkException: Failed to execute http call at com.apollographql.apollo.internal.interceptor.ApolloServerInterceptor$1$1.onFailure(ApolloServerInterceptor.java:105) at okhttp3.RealCall$AsyncCall.execute(RealCall.java:148) at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607) at java.lang.Thread.run(Thread.java:761) Caused by: java.net.SocketTimeoutException: timeout at okhttp3.internal.http2.Http2Stream$StreamTimeout.newTimeoutException(Http2Stream.java:593) at okhttp3.internal.http2.Http2Stream$StreamTimeout.exitAndThrowIfTimedOut(Http2Stream.java:601) at okhttp3.internal.http2.Http2Stream.takeResponseHeaders(Http2Stream.java:146) at okhttp3.internal.http2.Http2Codec.readResponseHeaders(Http2Codec.java:120) at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:75) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:120) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) at com.amazonaws.mobileconnectors.appsync.sigv4.AppSyncSigV4SignerInterceptor.intercept(AppSyncSigV4SignerInterceptor.java:182) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) at com.amazonaws.mobileconnectors.appsync.retry.RetryInterceptor.intercept(RetryInterceptor.java:48) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:185) at okhttp3.RealCall$AsyncCall.execute(RealCall.java:135) at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)  at java.lang.Thread.run(Thread.java:761) 

[scb01]

@danielvtan @donaldarmstrong

I am investigating this issue currently. My working theory at this point is that the mutation is being removed from the queue before it is successfully processed as reported in #40 and is lost if the operation fails due to a network error.

I am testing a couple of fixes locally that I hope to include in an upcoming release of the SDK. I will post back on this thread once I have more updates.

[scb01]

@danielvtan @donaldarmstrong

Version 2.7.1 of the SDK contains the queuing fixes logic to address the issue that you reported. Please upgrade to this version and let us know how you fare.

[scb01]

@danielvtan @donaldarmstrong
Looks like you are good to go. I'll close out this thread - please feel free to post back here or create a new thread if you are still running into issues.

[danielvtan]

@cbommas thanks for the fixes. Currently i can not test this as we are using a modified version of the sdk. We will test this for our next project.