Debugging 504 Gateway Time-out

[sachinkariyattin]

Debugging 504 Gateway Time-out

[kamilkisiela]

It's not enough information for me to help you solve the issue. Please share also logs from tokens service and some information about how you deploy / configure things.

[sachinkariyattin]

@kamilkisiela All the services are up and running successfully based on the logs. I am trying to deploy this in AWS. Below are the logs for tokens service. I am following the self hosting guide

{
    "level": 30,
    "time": 1712866137724,
    "pid": 7,
    "hostname": "ip-10-0-xx-xxx.ec2.internal",
    "msg": "Service \"tokens\" is ready"
}
{
    "level": 30,
    "time": 1712866137725,
    "pid": 7,
    "hostname": "ip-10-0-xx-xx.ec2.internal",
    "msg": "Server listening at http://[::]:8080"
}
2024-04-11T20:08:57.734Z ioredis:redis status[10.0.xxx.111:6379]: connecting -> connect
2024-04-11T20:08:57.734Z ioredis:redis status[10.0.xxx.111:6379]: connect -> ready
{
    "level": 30,
    "time": 1712866137735,
    "pid": 7,
    "hostname": "ip-10-0-xxx-xxx.ec2.internal",
    "msg": "Redis connection established"
}
{
    "level": 30,
    "time": 1712866137735,
    "pid": 7,
    "hostname": "ip-10-0-xxx-xxx.ec2.internal",
    "msg": "Redis is ready, purging the in-memory cache"
}
{
    "level": 30,
    "time": 1712866137735,
    "pid": 7,
    "hostname": "ip-10-0-xxx-xxx.ec2.internal",
    "msg": "Redis connection ready... "
}

I dont see any requests being logged in the tokens service though. This was for an older version of Hive. I tried deploying the latest images but encountering a different error. I have added the logs there as well. Please take a look at that issue as well

[sachinkariyattin]

@kamilkisiela

Tried with the latest version of Hive. Here is the error I am seeing in the server logs after the request gets timed out. I am using the token generated from the UI
Hive CLI: @graphql-hive/cli/0.34.1 darwin-x64 node-v18.19.1

TRPCClientError: fetch failed
    at TRPCClientError.from (file:///usr/src/app/@hive/server/index.js:283979:12)
    at file:///usr/src/app/@hive/server/index.js:284126:30 {
  message: 'Invalid token provided',
  path: [ 'schemaPublish' ],
  locations: [ { line: 2, column: 3 } ],
  extensions: [Object: null prototype] 
{}
}
TRPCClientError: fetch failed
    at TRPCClientError.from (file:///usr/src/app/@hive/server/index.js:283979:12)
    at file:///usr/src/app/@hive/server/index.js:284126:30 {
  message: 'Invalid token provided',
  path: undefined,
  locations: undefined,
  extensions: [Object: null prototype] 
{}

I am seeing other requests in the tokens service logs being processed

[200] (::ffff:10.0.102.21) GET /trpc/targetTokens?input=%7B%22targetId%22%3A%2291c19b68-89ee-46a2-ade3-994ff7e8697b%22%7D (reqId=2Buer+IaTk+SotEQzGrU4g/0000030104

Writing token (target=91c19b68-89ee-46a2-ade3-994ff7e8697b)

[200] (::ffff:10.0.102.21) POST /trpc/createToken (reqId=2Buer+IaTk+SotEQzGrU4g/0000030116)

But there are no logs for GET /trpc/getToken?

tokens.scopes field from the database

{target:registry:read,target:registry:write,target:read,project:read,organization:read}

[sachinkariyattin]

I was able to resolve this. If you are deploying this using the AWS redis cluster, ensure TLS is disabled by setting this flag to false.

This issue helped me get to the root cause.