Giter Site home page Giter Site logo

kaws's Introduction

Kaws

Named after the artist, kaws is a backend service that powers artsy.net collection pages. What are collections you ask? A Collection is a prefiltered version of Artsy's Collect page for marketing purposes.

Meta CircleCI codecov

Prerequisites

  • node.js 8.12.0 or newer
  • yarn 1.10.1 or newer
  • MongoDB 4.x

Note on Google API Access

KAWS exposes a yarn import-collections command that re-imports collection data from a specified Google spreadsheet. Invoke it like:

hokusai production run "yarn import-collections SPREADSHEET_ID SHEET_NAME"

...where SPREADSHEET_ID is from the Google spreadsheet's URL and SHEET_NAME is the name of the selected sheet. The ID must match the configured SPREADSHEET_IDS_ALLOWLIST list, and the following Google credentials are required:

  1. GOOGLE_API_KEY
  2. GOOGLE_CLIENT_ID
  3. GOOGLE_CLIENT_SECRET

See the hokusai and/or the Kaws Credentials secure note in 1passsword for current values of these.

If it becomes necessary to re-generate these credentials in staging and production — as has happened a couple of times in the past — you can follow these instructions.

For local development, you can run yarn test-google-config to see if your local .env file's values for GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET and SPREADSHEET_IDS_ALLOWLIST validate.

Getting started

First, make sure to install dependencies:

yarn

We use the .env file to load environment-specific variables. Copy .env.example to .env so dotenv will automatically load them upon app boot:

cp .env.example .env

Usually you do not have to change them, but if you need, update the variables to match your local development setup.

Once this is done, the next step would be to load some data.

Collections data can be pulled from the live staging or production environment using the following yarn command:

yarn pull-database staging # or production

Finally, start the server by running the command below:

yarn dev
# or
hokusai dev start

Then open http://localhost:4000/playground and you should see Apollo's GraphQL Playground. Try running the GraphQL query below, and if you see real collections data coming back from the database you are good to go!

query {
  collections {
    id
    slug
    title
  }
}

Testing

yarn jest
# or
hokusai test

Did you change the GraphQL schema?

Metaphysics is the current consumer of Kaws GraphQL schema, and keeps a copy of its latest schema in https://github.com/artsy/metaphysics/tree/main/src/data. If you have made changes to the Kaws GraphQL schema, make sure you also update the copy of this schema in Metaphysics. In order to do so follow these steps:

  1. In kaws run
yarn dump-schema
  1. copy the _schema.graphql file generated ☝🏼 to kaws.graphql
cp _schema.graphql kaws.graphql
  1. move the file above to your local Metaphysics under src/data and make a PR to Metaphysics with this change.

Updating the sitemap for Collect

When we update the production database with the update-database command, we also need to inform Google of the updates so newly added collections will be crawled. This could be done with the update-sitemap command. This comamnd needs to be run in the k8s container as it neeeds to point to the production databse:

hokusai production run 'yarn update-sitemap'

About Artsy

This project is the work of engineers at Artsy, the world's leading and largest online art marketplace and platform for discovering art. One of our core Engineering Principles is being Open Source by Default which means we strive to share as many details of our work as possible.

You can learn more about this work from our blog and by following @ArtsyOpenSource or explore our public data by checking out our API. If you're interested in a career at Artsy, read through our job postings!

kaws's People

Contributors

anandaroop avatar artsy-peril[bot] avatar artsyit avatar artsyjian avatar ashleyjelks avatar damassi avatar dblandin avatar dependabot-preview[bot] avatar dependabot[bot] avatar dimatretyak avatar eessex avatar icirellik avatar iskounen avatar izakp avatar joeyaghion avatar jonallured avatar jpotts244 avatar l2succes avatar mbilokonsky avatar mdole avatar mzikherman avatar ovasdi avatar pepopowitz avatar peril-staging[bot] avatar renovate-bot avatar starsirius avatar sweir27 avatar xtina-starr avatar yuki24 avatar zephraph avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

kaws's Issues

No hot reloading?

I was making changes to the schema and not seeing them reflected using yarn dev until I killed and restarted the process.

Allow `size` param to be specified for `collections`

This is so that when retrieving collections for KAWS (for example), we can just retrieve a few, and not all of them. Especially if you want to also retrieve artworks for each collection (hitting ES), we should be careful not to overfetch.

`yarn start` doesn't fully work in dev mode

I wasn't getting updates to the GraphQL Playground (even when restarting the server), when modifying the schema. Turns out I needed to run yarn dev instead of yarn start.

I think yarn start is the more conventional command to support.

Improve dev server start UX

After pairing with @xtina-starr on some kaws work yesterday, a few pieces of feedback came up:

  • Currently one needs to remember to run yarn build before running yarn start, which is too easy to forget. One should really only need to run yarn start and be up and running.
  • Because kaws has server code, you may want to leverage a start script, like the one in Force, so that when running from vscode’s terminal you automatically get the debugger connected.

Rewrite specs to not stub database

Currently, the database and related methods are being stubbed. Since we're having an integration test anyway that runs a query against the schema (so not super unit-y), we should be able to assert database functionality is working. So actually save and retrieve objetcs, etc.

Dependency Dashboard

This issue provides visibility into Renovate updates and their statuses. Learn more

Pending Approval

These branches will be created by Renovate only once you click their checkbox below.

  • chore(deps): pin dep algoliasearch from 4.10.5 to v
  • chore(deps): update dep @sentry/node from 4.6.4 to v4.6.6
  • chore(deps): update dep @types/elasticsearch from 5.0.36 to v5.0.40
  • chore(deps): update dep @types/lodash from 4.14.116 to v4.14.182
  • chore(deps): update dep @types/mongodb-uri from 0.9.0 to v0.9.1
  • chore(deps): update dep node-fetch from 2.6.1 to v2.6.7
  • chore(deps): update dep reflect-metadata from 0.1.12 to v0.1.13
  • chore(deps): update dep release-it from 7.6.1 to v7.6.3
  • chore(deps): update dep @types/node from 12.12.11 to v12.20.52
  • chore(deps): update dep algoliasearch from 4.10.5 to v4.13.1
  • chore(deps): update dep aws-sdk from 2.1033.0 to v2.1137.0
  • chore(deps): update dep body-parser from 1.18.3 to v1.20.0
  • chore(deps): update dep csv-parser from 2.1.0 to v2.3.5
  • chore(deps): update dep danger from 4.0.2 to v4.4.10
  • chore(deps): update dep dd-trace from 0.6.0 to v0.36.3 (dd-trace, @types/dd-trace)
  • chore(deps): update dep dotenv from 8.2.0 to v8.6.0
  • chore(deps): update dep elasticsearch from 15.3.1 to v15.5.0
  • chore(deps): update dep husky from 1.1.2 to v1.3.1
  • chore(deps): update dep mongodb from 3.6.0 to v3.7.3
  • chore(deps): update dep morgan from 1.9.1 to v1.10.0 (morgan, @types/morgan)
  • chore(deps): update dep nodemon from 1.18.10 to v1.19.4
  • chore(deps): update dep prettier from 2.1.2 to v2.6.2
  • chore(deps): update dep slugify from 1.3.4 to v1.6.5
  • chore(deps): update dep tslint from 6.0.0 to v6.1.3
  • chore(deps): update dep tslint-config-prettier from 1.15.0 to v1.18.0
  • chore(deps): update dep typeorm from 0.2.25 to v0.3.6
  • chore(deps): update dep @sentry/node from 4.6.4 to v6
  • chore(deps): update dep @types/jest from 23.3.14 to v27
  • chore(deps): update dep @types/node from 12.12.11 to v17
  • chore(deps): update dep csv-parser from 2.1.0 to v3
  • chore(deps): update dep danger from 4.0.2 to v11
  • chore(deps): update dep dd-trace from 0.6.0 to v2
  • chore(deps): update dep dotenv from 8.2.0 to v16
  • chore(deps): update dep elasticsearch from 15.3.1 to v16
  • chore(deps): update dep googleapis from 92.0.0 to v100
  • chore(deps): update dep graphql-yoga from 1.18.3 to v2
  • chore(deps): update dep husky from 1.1.2 to v8
  • chore(deps): update dep jest from 23.6.0 to v28
  • chore(deps): update dep lint-staged from 7.3.0 to v12
  • chore(deps): update dep mongodb from 3.6.0 to v4
  • chore(deps): update dep node-fetch from 2.6.1 to v3
  • chore(deps): update dep nodemon from 1.18.10 to v2
  • chore(deps): update dep release-it from 7.6.1 to v15
  • chore(deps): update dep ts-jest from 23.10.5 to v28
  • chore(deps): update dep ts-node from 7.0.1 to v10
  • chore(deps): update dep turndown from 5.0.3 to v7
  • chore(deps): update dep type-graphql from 0.17.6 to v1

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

circleci
.circleci/config.yml
  • codecov 1.2.5
  • hokusai volatile
  • yarn 5.1.3
  • horizon volatile
npm
package.json
  • @sentry/node 4.6.4
  • algoliasearch ^4.10.5
  • aws-sdk 2.1033.0
  • body-parser 1.18.3
  • class-transformer 0.5.1
  • csv-parser 2.1.0
  • dd-trace 0.6.0
  • elasticsearch 15.3.1
  • googleapis 92.0.0
  • graphql-yoga 1.18.3
  • lodash 4.17.21
  • mongodb 3.6.0
  • mongodb-uri 0.9.7
  • morgan 1.9.1
  • node-fetch 2.6.1
  • reflect-metadata 0.1.12
  • slugify 1.3.4
  • turndown 5.0.3
  • type-graphql 0.17.6
  • typeorm 0.2.25
  • @types/dd-trace 0.6.1
  • @types/elasticsearch 5.0.36
  • @types/jest 23.3.14
  • @types/lodash 4.14.116
  • @types/mongodb-uri 0.9.0
  • @types/morgan 1.7.35
  • @types/node 12.12.11
  • bash-color 0.0.4
  • danger 4.0.2
  • dotenv 8.2.0
  • husky 1.1.2
  • jest 23.6.0
  • lint-staged 7.3.0
  • nodemon 1.18.10
  • prettier 2.1.2
  • release-it 7.6.1
  • ts-jest 23.10.5
  • ts-node 7.0.1
  • tslint 6.0.0
  • tslint-config-prettier 1.15.0
  • typescript 4.6.4
  • node 12

  • Check this box to trigger a request for Renovate to run again on this repository

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.