graphile/gatsby-source-pg

PostgreSQL source for Gatsby, powered by PostGraphile

gatsby-source-pg's Issues

Get nodes created by plugin by their type

If I explore data, created by plugin, I can find:

postgres (PostGraphile) {
    allElements (PostGraphile_ElementsConnection) {
      nodes (PostGraphile_Element) {
        ....
      }
    }
  }

And I even can modify these nodes like this:

exports.createResolvers = ({ createResolvers }) => {
    createResolvers({
    PostGraphile_Element: {
    internal: {
        type: `Internal`,
        resolve: object => {
            return {
                type: `pgData`,
            }
        }
    }
  },
});
}

But if I try to use this node in other places with constructions:

if (node.internal.type === `pgData`) {
       // do smth..
    }

const nodes =  getNodesByType(`pgData`)) || getNodesByType(`PostGraphile_Element`)) 

I get nothing.

So, If I want to build lunr index from nodes with lunr-plugin I try to use such syntax, as described in its rtfm for gatsby-config.js:

{
    resolve: `@gatsby-contrib/gatsby-plugin-elasticlunr-search`,
    options: {
      fields: [`name`],
      // How to resolve each field`s value for a supported node type
      resolvers: {
        PostGraphile_Element: {
        name: node => node.name,
        }, 
        // Or try to use my crated internal.type:
        pgData: {
          name: node => node.name,
        },
      },
    },
  },

The question is for I can get created child nodes by their type PostGraphile_Element or pgData

Error on connecting

Thank you for the plugin I try to connect my database I got this source and transform nodes - 803.113s
and when I remove schema: public, its build but with out connect to server
{ resolve: gatsby-source-pg, options: { connectionString: postgres://username@localhost:5432/db, schema: public, refetchInterval: 60, // Refetch data every 60 seconds }, },

Error: Unexpected schema type: PostGraphile Query

1 - Cloning` the example repo
2 - Updating packages to the latest versions
3 - Running gatsby build
I get

 ERROR #11321  PLUGIN

"gatsby-source-pg" threw an error while running the sourceNodes lifecycle:

Unexpected schema type: PostGraphileQuery

  109 |   };
  110 | 
> 111 |   const schema = wrapSchema({
      |                  ^
  112 |     schema: introspectionSchema,
  113 |     executor: linkToExecutor(link)
  114 |   }, [new StripNonQueryTransform(), new RenameTypes(name => `${typeName}_${name}`), new NamespaceUnderFieldTransform({

File: node_modules/gatsby-source-graphql/gatsby-node.js:111:18



  Error: Unexpected schema type: PostGraphileQuery
  
  - rewire.js:96 rewireNamedType
    /home/tareef/gatsby-source-pg-example/dist/utils/src/rewire.js:96:15
  
  - rewire.js:22 
    /home/tareef/gatsby-source-pg-example/dist/utils/src/rewire.js:22:32
  
  - Array.forEach
  
  - rewire.js:21 rewireTypes
    /home/tareef/gatsby-source-pg-example/dist/utils/src/rewire.js:21:29
  
  - mapSchema.js:30 mapSchema
    /home/tareef/gatsby-source-pg-example/dist/utils/src/mapSchema.js:30:37
  
  - clone.js:41 cloneSchema
    /home/tareef/gatsby-source-pg-example/dist/utils/src/clone.js:41:12
  
  - transforms.js:3 
    /home/tareef/gatsby-source-pg-example/dist/utils/src/transforms.js:3:115
  
  - Array.reduce
  
  - transforms.js:3 Object.applySchemaTransforms
    /home/tareef/gatsby-source-pg-example/dist/utils/src/transforms.js:3:23
  
  - generateProxyingResolvers.js:22 generateProxyingResolvers
    [gatsby-source-pg-example]/[gatsby-source-graphql]/dist/wrap/src/generateProxyingResolve    rs.js:22:31
  
  - wrapSchema.js:20 wrapSchema
    [gatsby-source-pg-example]/[gatsby-source-graphql]/dist/wrap/src/wrapSchema.js:20:31
  
  - gatsby-node.js:111 Object.exports.sourceNodes
    [gatsby-source-pg-example]/[gatsby-source-graphql]/gatsby-node.js:111:18
  
  - task_queues.js:97 processTicksAndRejections
    internal/process/task_queues.js:97:5
  
  - gatsby-node.js:24 Object.exports.sourceNodes
    [gatsby-source-pg-example]/[gatsby-source-pg]/gatsby-node.js:24:3
  
  - api-runner-node.js:283 runAPI
    [gatsby-source-pg-example]/[gatsby]/src/utils/api-runner-node.js:283:16

https://github.com/tareefdev/gatsby-source-pg-example

How to use pagination?

I understand how to use pagination in GraphQL query but how Gatsby is going to generate all the pages with N items per page?

Update to graphql 16

Summary

Please update the package so we can use Gatsby v5. I guess it is necessary to update the graphql dependency from:

"graphql": "^0.13.x | ^14.0.0 | ^15.0.0",

to also be compatible with graphl 16 since Gatsby uses graphl ^16.6.0

Dependency ambiguous

Summary

npm: Cannot find module '@apollo/client/link/core'
yarn: Error: @graphql-tools/links tried to access @apollo/client (a peer dependency) but it isn't provided by its ancestors; this makes the require call ambiguous and unsound. Required package: @apollo/client (via "@apollo/client/link/core") Required by: @graphql-tools/links@virtual:97b3f984b9964688474bddf6446ba7eccd800a7b5f951460466b2327fab63d2bd1e398fb322d1530587a80f6699f81b4730cc76d53c6c0863f46d7e6d5273a71#npm:7.1.0

Versions

gatsby 3.10.1
gatsby-source-pg 0.2.1
node v16.5.0
yarn-2.4.2
npm 7.20.0

Please Upgrade (And Convert to Peer Dependency?) PG Module In gatsby-source-pg

Description

Build hangs forever on the:

source and transform nodes

step. I've confirmed that an update to the pg dependency resolves the issue.

Steps to reproduce

This didn't actually help my issue, but since updating dependencies is pretty painless, and since per brianc/node-postgres#2069 hanging issues (for other people) can be fixed with an update:

You should upgrade to the latest pg (8.3.0) instead of downgrading to an EOL Node.

It seems like it'd be a nice quick change that might help others.

Actually, although at first I thought a pg version update didn't fix things, further investigation found that it did, and that this is a manifestation of a known pg issue. A maintainer from that project helped me realize it (backstory here: brianc/node-postgres#2300).

Expected result

What should happen?

The plug-in should not hang.

Actual result

It hangs.

Environment

System:
OS: Linux 5.4 Linux Mint 20 (Ulyana)
CPU: (8) x64 Intel(R) Core(TM) i7-4810MQ CPU @ 2.80GHz
Shell: 5.0.17 - /bin/bash
Binaries:
Node: 14.7.0 - /usr/bin/node
npm: 6.14.7 - /usr/bin/npm
Browsers:
Chrome: 84.0.4147.105
Firefox: 78.0.2
npmPackages:
gatsby: ^2.24.15 => 2.24.15
gatsby-cli: ^2.12.47 => 2.12.47
gatsby-image: ^2.4.7 => 2.4.7
gatsby-plugin-import: ^2.1.5 => 2.1.5
gatsby-plugin-manifest: ^2.4.11 => 2.4.12
gatsby-plugin-modal-routing: ^1.2.0 => 1.2.0
gatsby-plugin-offline: ^3.2.9 => 3.2.11
gatsby-plugin-react-helmet: ^3.3.4 => 3.3.4
gatsby-plugin-root-import: ^2.0.5 => 2.0.5
gatsby-plugin-s3: ^0.3.4 => 0.3.4
gatsby-plugin-sharp: ^2.6.11 => 2.6.12
gatsby-source-filesystem: ^2.3.11 => 2.3.12
gatsby-source-pg: ^0.1.4 => 0.1.4
gatsby-transformer-sharp: ^2.5.5 => 2.5.5
npmGlobalPackages:
gatsby: 2.24.15

Expected value of type "PostGraphile" but got: {}

gatsby-source-pg@^0.1.4

Connection string provided to a remote database (running on my synology, not local). http://localhost:8000/___graphql lists queries for all my databases which leads me to believe I've got my connection correct.

When executing a query in __graphql, I get the following error:

   "stack": [
        "GraphQLError: Expected value of type \"PostGraphile\" but got: {}.",
        "    at invalidReturnTypeError (/Volumes/Code/personal/project/node_modules/graphql/execution/execute.js:707:10)",
        "    at completeObjectValue (/Volumes/Code/personal/project/node_modules/graphql/execution/execute.js:699:13)",
        "    at completeValue (/Volumes/Code/personal/project/node_modules/graphql/execution/execute.js:591:12)",
        "    at completeValue (/Volumes/Code/personal/project/node_modules/graphql/execution/execute.js:557:21)",
        "    at /Volumes/Code/personal/project/node_modules/graphql/execution/execute.js:492:16",
        "    at processTicksAndRejections (internal/process/task_queues.js:97:5)",
        "    at async Promise.all (index 0)"
      ]

It's a clean project otherwise. I created a new gatsby project and installed this plugin. I tried to add a simple query to the default index.js page.

Migrating from JSON to Postgres + gatsby-source-pg

I'm trying to migrate from JSON file sourse to postgresql. And faced some issues with gatsby-source-pg:

1. under_score was replaced to camelCase. Can this option be turned off in config?
2. plugins, like @gatsby-contrib/gatsby-plugin-elasticlunr-search and gatsby's onCreateNode widely use nodes attribytes (eg. node.internal.type, resolver names). But plugin gives only PostGraphile_NAME type of node element whish is not suitable for that purposes. How it can be fixed?

Not working with gatsby 2.2.11

I created a fresh project with gatsby new . Got some warnings as below -

warning "gatsby > [email protected]" has incorrect peer dependency "graphql@^0.12.0 || ^0.13.0".
warning "gatsby > [email protected]" has incorrect peer dependency "graphql@^0.10.0 || ^0.11.0 || ^0.12.0 || ^0.13.0".
warning "gatsby > [email protected]" has incorrect peer dependency "graphql@^0.13.0".

But this did not prevent me using GraphQL for siteMetadata queries. But as soon as I added this plugin for my Postgres data GraphiQL playground started complaining as below error -

"message": "Cannot use GraphQLSchema "[object GraphQLSchema]" from another module or realm.

Ensure that there is only one instance of "graphql" in the node_modules directory. If different versions of "graphql" are the dependencies of other relied on modules, use "resolutions" to ensure only one version is installed.

https://yarnpkg.com/en/docs/selective-version-resolutions

Duplicate "graphql" modules cannot be used at the same time since different versions may have different capabilities and behavior. The data from one version used in the function from another could produce confusing and spurious results.",

Because of above error, neither GraphiQL nor application is getting any data from Postgres

My environment:
Node - v11.6.0
gatsby-cli - v2.4.17
gatsby - v2.2.11
gatsby-source-pg - v0.1.2

Please advise. Thanks.

The graphql version is forced to be 0.13 while gatsby 2.3.33 require 14.2.1

This is similar to #3 but the solution to add

  "resolutions": {
    "graphql": "14.x"
  }

to the package.json and then run

rm -r node_modules                                    
npx npm-force-resolutions
npm install

Does not help as I'm still getting old graphql

 npm ls graphql                                      
[email protected] ./pwcvis
├─┬ [email protected]
│ └── [email protected] 
├─┬ [email protected]
│ ├── [email protected] 
│ └─┬ [email protected]
│   └── [email protected] 
└── [email protected] 

Option to Not Disable Mutations

I realize that in most Gatsby apps there's no need for mutations, but what if you want to be able to edit your database from your dev server (before deploying ... ie. if you want to create a mini-CMS on your dev server)?

It seems like Gatsby/Gatsby-source-pg should be fully capable of serving mutations, since it just leverages Postgraphile, and that tool certainly can make mutations ... but if I just try to change createSchema.js like so:

const skipPlugins = [
  graphileBuild.QueryPlugin,
  // graphileBuild.MutationPlugin,
  graphileBuild.SubscriptionPlugin,
];

the queries stop working, presumably because of whatever is going on in RenamedQueryPlugin.

I'd be happy to submit a PR with a RenamedMutationPlugin and whatever else is needed to add some sort of enableMutations: true option, if A) someone could take a quick sec to explain what's going on there, and B) if such a PR would actually be desired.

If it's just a goal to keep this plug-in read-only then I guess I'll just have to run a duplicate/parallel Postgraphile server (which isn't the end of the world, just sub-optimal).

gatsby-source-pg incompatible with gatsby 4

Summary

gatsby-source-pg incompatible with gatsby 4

gatsby build:

Calling addThirdPartySchema in the sourceNodes API is deprecated. Please use: createSchemaCustomization

There was an error in your GraphQL query:

Cannot query field "postgres" on type "Query".

Source image as a dynamic data

I am looking into sourcing images via PG as a field. I was trying to investigate if there is a simple, easy way to achieve this with some simple code, but currently -- I can't really find one.

Before I will dig in into creating a graphile plugin to handle this, can someone direct me to a ready-to-use solution?

Connecting to Heroku

Summary

I'd like to connect to a Postgres instance running in Heroku.

Additional context

I use Heroku's DATABASE_URL environment variable, with the suffix ?ssl=1 in my connectionString. When I run gatsby develop I get this error:

[client] success onPreBootstrap - 0.015s
[client] success createSchemaCustomization - 0.006s
[client]
[client]  ERROR #11321  PLUGIN
[client]
[client] "gatsby-source-pg" threw an error while running the sourceNodes lifecycle:
[client]
[client] self signed certificate
[client]
[client]
[client]
[client]   Error: self signed certificate
[client]
[client]   - From previous event:
[client]
[client]   - api-runner-node.js:460 Promise.catch.decorateEvent.pluginName
[client]     [yl]/[gatsby]/src/utils/api-runner-node.js:460:9
[client]
[client]   - From previous event:
[client]
[client]   - api-runner-node.js:459
[client]     [yl]/[gatsby]/src/utils/api-runner-node.js:459:14
[client]
[client]   - timers.js:456 processImmediate
[client]     internal/timers.js:456:21
[client]
[client]   - From previous event:
[client]
[client]   - api-runner-node.js:451
[client]     [yl]/[gatsby]/src/utils/api-runner-node.js:451:13
[client]
[client]   - From previous event:
[client]
[client]   - api-runner-node.js:367 module.exports
[client]     [yl]/[gatsby]/src/utils/api-runner-node.js:367:3
[client]
[client]   - source-nodes.ts:97 _default
[client]     [yl]/[gatsby]/src/utils/source-nodes.ts:97:9
[client]
[client]   - source-nodes.ts:24 sourceNodes
[client]     [yl]/[gatsby]/src/services/source-nodes.ts:24:9
[client]
[client]   - interpreter.js:707 Interpreter.exec
[client]     [yl]/[xstate]/lib/interpreter.js:707:27
[client]
[client]   - interpreter.js:210 Interpreter.execute
[client]     [yl]/[xstate]/lib/interpreter.js:210:22
[client]
[client]   - interpreter.js:230 Interpreter.update
[client]     [yl]/[xstate]/lib/interpreter.js:230:18
[client]
[client]   - interpreter.js:131
[client]     [yl]/[xstate]/lib/interpreter.js:131:23
[client]
[client]   - scheduler.js:60 Scheduler.process
[client]     [yl]/[xstate]/lib/scheduler.js:60:13
[client]
[client]   - scheduler.js:44 Scheduler.schedule
[client]     [yl]/[xstate]/lib/scheduler.js:44:14
[client]
[client]   - interpreter.js:127 Interpreter.send
[client]     [yl]/[xstate]/lib/interpreter.js:127:29
[client]
[client]   - interpreter.js:825 actor.id
[client]     [yl]/[xstate]/lib/interpreter.js:825:23
[client]
[client]
[client] warn The gatsby-source-pg plugin has generated no Gatsby nodes. Do you need it?

add more possibilities into condition tests

Hi, i couldn't find an answer so ;aybe it's not possible right now>

Is it possible to use condition other than equality in condition ?
i assume you are using this packages so it should be possible ? postgraphile-plugin-connection-filter

it would be nice to do something like this for example :

{
  postgres {
    allPosts(condition: {createdAt: ["<=", "2019-01-07"] }) {
      totalCount
    }
  }
}

{
  postgres {
    allPosts(condition: {createdAt: [ ["<=", "2019-01-07"], [">=", "2018-12-07"] ] }) {
      totalCount
    }
  }
}

do you think that would be possible ?

support for mutations?

This is an awesome plugin! Thank you for bringing postgraphile to gatsby!

I am wondering if there is any support for mutations or plans to bring them to this plugin.

gatsby-node.js Should Pass a `connectionTimeoutMillis` When Creating the Connection Pool

Currently if anything goes wrong while trying to create the database connection, Gatsby will just hang on the "source and transform nodes" step.

This happens because of this code in gatsby-node.js:

  const pool = new Pool({
    connectionString
  });

Since no connectionTimeoutMillis option is specified, if the connection fails Node will just hang. If you instead provided a limit, for instance ten seconds ...

  const pool = new Pool({
    connectionString,
    connectionTimeoutMillis: 10 * 1000,
  });

users would get a helpful error at their console telling them that they have some sort of database connection issue ... instead of just sitting with no feedback forever.