commercetools/commercetools-sunrise-data

The data (product types, categories, products) for Sunrise

Sunrise Data

Prerequisites

1. Access to a commercetools Project and the Merchant Center. If you do not have a commercetools Project, follow our Getting started guide.
2. Node.js (version 10 or later) must be installed on your computer.

Setting up your Sunrise Data project

1. Clone this repository to your computer.
2. Go to the root of the project, where package.json is located, and install all node dependencies with:

npm install

3. Create an API client in the Merchant Center.
4. Download the Environment Variables (.env) file.

6. Rename this file .env and move it to the root of the project.

Commands

1. Clean all existing Project data and import new:

       npm run start
   

2. Clean project data:

       npm run clean:data
   

3. Import Project data:

       npm run import:data
   

4. Clean or import certain data (e.g. Categories, Products, Customers, etc.)

       npm run clean:categories
   

   or

       npm run import:products
   

   or

       npm run import:customers
   

commercetools-sunrise-data's Issues

master variant sku is missing

Delete products without images

There are ~ 300 products where in no variant is an image.
Can I delete them @trangng @CT-Szilvia ?

Move to Heroku cedar 14

drop unused attribute definitions

• @stmeissner reported lookProducts is unused

use product names for suggest keywords

central "WEB" root category for categories

• would also be cool if the categories would have a central "WEB" root category instead of starting directly in the built-in root. That makes various things easier, too.
• if you put the regular category tree in a special root you still have the chance to do special cases like landing pages outside that root.

remove some prices out of the product

Discuss style of job/steps parameters

It can be a better style (for IDE compatibility and other issues) to get the beans via methods call instead of parameter.

Provide an option to import smaller data that fit the trial project restrictions

Our Trial mode projects are limited (e.g. 5k products), which is why this importer fails hard.

It would be good to have an option to import only a subset that fits trial projects.

iron.io integration

sample: https://github.com/mikkoi/io.iron.ironworker.sendgrid.simple

Implement missing tests

Delete unused classes

e.g. CommercetoolsPayloadFileConfig

english slugs should only contain english words

German terms in English names and slugs

"productId": "b373d004-9914-4e2f-bec0-7f1e2420a072",
"name": {
"en": "Michael Kors – handtaschen “Jet Set Travel”",
"de": "Michael Kors – Handtasche „Jet Set Item”",
"it": "Michael Kors – Borsa a mano “Jet Set Travel”"
},
"productSlug": {
"de": "michaelkors-handtasche-30F2GTTT8L-braun",
"en": "michaelkors-handtaschen-30F2GTTT8L-brown",
"it": "michaelkors-borsaamano-30F2GTTT8L-marrone"
},

add more product images per product

Clean up code from printing

Use logger instead.

the products don't have a tax category assigned so they cannot added to a cart

travis build

• with all data
• with limit to 1000

Enhance data for the US market

• We need a $ price for the US

Have better attribute names

The product attribute names are displayed directly in the shop, so they should be nicer.
For example: Color instead of Filtercolor or Size instead of Filtersize

CommercetoolsJobConfiguration should be used via @Import

Not as inheritance.

Create official job for Tax Category

Currently it is hidden inside ProductImportJob.

Have images as Assets

CSV reader able to read attributes from header

Instead of having them hardcoded into the code.

import app does not publish products

Create official job for customer group import

Currently it is hidden inside another job (ProductsImportJob).

Image URLs should not include protocol

instead of

https://images.example.com/foo.jpg

we should use

//images.example.com/foo.jpg

check the parent categories for a product

• for the sunrise data it would be really helpful if we could check the parent categories for a product as we are not using the the category folder as a landing page but as a filter funnel
• to leave out the parent categories makes only sense when the category directory is a seo landing page showing specific products, which is not the case for sunrise

Improve CategoryImportJob to avoid additional call for parent category

Remove joyride references and replace for normal availability/prices/channels

test HipChat notification

Add missing category description

description is missing

allow to set a param to disable SSL cert validation

for our staging we have self signed certificate and it would be nice to enable to use the importer on staging, by default the SSL certs should be validated

Enable channels/prices/availability import for iron.io

Automated release to iron.io
Can use: https://github.com/commercetools/iron-maven-plugin

Fix product category relationship

• There are a lot of products without a reference to categories.
• Some products need to be in the sales categories
• Some products need to be in the NEW categories

Make field "manufacturer AID" not searchable

The manufacturer ID fields seems to hit pretty hard in the search and some products have ambiguous values. E.g. a shoe has "jeans" in the manufacturer ID.

generally, fields that are not visible in the webshop should not be searchable. https://demo.commercetools.com/en/search?q=jeans

make it easy and painless to import the data automatically

• make gap analysis what's missing
• fill gaps
• make it easy to install all necessary tools in one go
• clearly document process
• delete incorrect stuff

add Maven wrapper to imprtApp

Use separate storage for images and use CDN URLs in product data.

Provide category description

Make all imports resilient to failures

Check last one modified and continue from that.

• Products
• Categories
• Product Types
• Channels
• Prices
• Inventory
• Customer Group
• Product Suggestions

productType JSON file cannot be used as payload for create method

unlike explained in the Readme.

The reason is that the /product-types/product-type.json file contains a ProductType instead of the respective draft.

Category "Wies'n Special" missing in categories.csv

During product import via IMPEX following error message is thrown:

[ '[row 3676:categories] Can not find category for 'Wies'n Special'!',
'[row 8884:categories] Can not find category for 'Wies'n Special'!',
'[row 9551:categories] Can not find category for 'Wies'n Special'!',
'[row 14306:categories] Can not find category for 'Wies'n Special'!' ]
[ '[row 3676:categories] Can not find category for 'Wies'n Special'!',
'[row 8884:categories] Can not find category for 'Wies'n Special'!',
'[row 9551:categories] Can not find category for 'Wies'n Special'!',
'[row 14306:categories] Can not find category for 'Wies'n Special'!' ]

Document how to import availability/prices/channels

Currently it is below in a separated category "joyride".

Remove empty categories

• Women/Shoes/Boots
• Women/Shoes/Lace-Up Shoes
• Women/Bags/Bucketbag & Packback
• Women/Looks (not even accessible right now)
• Men/Bags (and subcategories)
• Men/Shoes/Boots
• Men/Shoes/Sandals
• Men/Looks (not even accessible right now)
• Accessories/Women/Shoes
• Accessories/Women/Bags
• Accessories/Men/Sunglasses
• Brands
• New/Women/Looks
• New/Men/Bags
• New/Men/Looks
• New/Special (and subcategories)

All category names should be of format 'Name'

first letter upper case, rest lower

Clean up annotation

We may have overused them a bit :)

Wrong pics

These pics fail sometimes, others always. It breaks the layout.

https://sphere-sunrise-dmexco.herokuapp.com/en/allstar-converse-sneakers-1C332-grey-M0E20000000EBBX.html
https://sphere-sunrise-dmexco.herokuapp.com/en/altea-scarf-1550670-grey-A0E200000002512.html
https://sphere-sunrise-dmexco.herokuapp.com/en/anaalcazar-dress-0441541887-multi-M0E20000000EAO3.html
https://sphere-sunrise-dmexco.herokuapp.com/en/blauer-downjacket-BLUC037753214-green-M0E20000000E5M3.html
https://sphere-sunrise-dmexco.herokuapp.com/en/dkny-handbag-r1514001-beige-A0E2000000026HN.html
https://sphere-sunrise-dmexco.herokuapp.com/en/dondup-chino-asly-PDH999-black-M0E20000000E0AZ.html
https://sphere-sunrise-dmexco.herokuapp.com/en/dondup-chino-gaubert-UP235EF039U-grey-M0E20000000E0GO.html
https://sphere-sunrise-dmexco.herokuapp.com/en/dondup-jeans-monroe-P692DS107DG61-blue-M0E20000000E060.html
https://sphere-sunrise-dmexco.herokuapp.com/en/elisabetta-franchi-dress-AB8833121-white-M0E20000000EHOP.html
https://sphere-sunrise-dmexco.herokuapp.com/en/elisabetta-franchi-trousers-pa7673881-multi-M0E20000000EHSB.html
https://sphere-sunrise-dmexco.herokuapp.com/en/gabs-bag-DAPHNE-1201M-yellow-A0E200000002B7C.html
https://sphere-sunrise-dmexco.herokuapp.com/en/gabs-wallet-gmoney17-1101-cream-A0E200000001WOY.html
https://sphere-sunrise-dmexco.herokuapp.com/en/guess-bag-jizelle-mini-52500-blue-A0E200000001YXH.html
https://sphere-sunrise-dmexco.herokuapp.com/en/guess-wallet-48640-black-A0E200000002845.html
https://sphere-sunrise-dmexco.herokuapp.com/en/guess-wallet-blossom-medium-78640-black-A0E200000001YTY.html
https://sphere-sunrise-dmexco.herokuapp.com/en/guess-wallet-blossom-medium-78640-multi-A0E200000001YTX.html
https://sphere-sunrise-dmexco.herokuapp.com/en/guess-wallet-blossom-medium-78640-red-A0E200000001YU0.html
https://sphere-sunrise-dmexco.herokuapp.com/en/guess-wallet-blossom-medium-78640-white-A0E200000001YU1.html
https://sphere-sunrise-dmexco.herokuapp.com/en/guess-wallet-blossom-small-78560-black-A0E200000001YTT.html
https://sphere-sunrise-dmexco.herokuapp.com/en/guess-wallet-blossom-small-78560-grey-A0E200000001YTU.html
https://sphere-sunrise-dmexco.herokuapp.com/en/guess-wallet-blossom-small-78560-red-A0E200000001YTV.html
https://sphere-sunrise-dmexco.herokuapp.com/en/guess-wallet-blossom-small-78560-white-A0E200000001YTW.html
https://sphere-sunrise-dmexco.herokuapp.com/en/guess-wallet-greyson-30460-black-A0E200000001YW3.html
https://sphere-sunrise-dmexco.herokuapp.com/en/guess-wallet-greyson-30460-white-A0E200000001YWF.html
https://sphere-sunrise-dmexco.herokuapp.com/en/guess-wallet-jinha-29460-blue-A0E200000001YUT.html
https://sphere-sunrise-dmexco.herokuapp.com/en/guess-wallet-jinha-29460-pink-A0E200000001YUU.html
https://sphere-sunrise-dmexco.herokuapp.com/en/guess-wallet-jinha-29460-white-A0E200000001YUV.html
https://sphere-sunrise-dmexco.herokuapp.com/en/guess-wallet-logoatheart-small-lightgrey-A0E200000001YSZ.html
https://sphere-sunrise-dmexco.herokuapp.com/en/guess-wallet-paxton-15460-multi-rose-A0E200000001YV1.html
https://sphere-sunrise-dmexco.herokuapp.com/en/himons-shirt-8150-beige-M0E20000000E8MU.html
https://sphere-sunrise-dmexco.herokuapp.com/en/kanna-espadrille-KV5511-black-white-M0E20000000EIY8.html
https://sphere-sunrise-dmexco.herokuapp.com/en/michael-kors-bag-30S5SCPS3L-lightblue-A0E2000000027C8.html
https://sphere-sunrise-dmexco.herokuapp.com/en/michael-kors-bag-30S5SGRT6U-white-A0E2000000027C9.html
https://sphere-sunrise-dmexco.herokuapp.com/en/michael-kors-bag-jetsettravel-32F4GTVC1M-gold-A0E200000001YKG.html
https://sphere-sunrise-dmexco.herokuapp.com/en/michael-kors-shopper-30T3STVT6L-lightblue-A0E2000000027CE.html
https://sphere-sunrise-dmexco.herokuapp.com/en/michael-kors-shoulderbag-30T3SLMM2L-lightblue-A0E2000000027CB.html
https://sphere-sunrise-dmexco.herokuapp.com/en/moorer-coat-CIPROOP-darkblue-M0E20000000DJ8X.html
https://sphere-sunrise-dmexco.herokuapp.com/en/philippemodel-sneakers-white-flowers-M0E20000000DPNR.html
https://sphere-sunrise-dmexco.herokuapp.com/en/pinko-jacket-1G10RNY16RH26-yellow-M0E20000000DJIX.html
https://sphere-sunrise-dmexco.herokuapp.com/en/sam-edelman-sandalen-D3118L1002-multi-M0E20000000DQG3.html
https://sphere-sunrise-dmexco.herokuapp.com/en/sam-edelman-sandals-adora-C7443L1001-black-M0E20000000DQDV.html
https://sphere-sunrise-dmexco.herokuapp.com/en/siviglia-chino-B2A2S2060150-darkblue-M0E20000000E253.html
https://sphere-sunrise-dmexco.herokuapp.com/en/stone-island-sweater-V0037-red-M0E20000000DSUF.html
https://sphere-sunrise-dmexco.herokuapp.com/en/twin-set-top-j2s5np-cream-M0E20000000E959.html

Remove internal ids from CSV files.