Giter Site home page Giter Site logo

catalog-storefront's Introduction

Overview

This repository is responsible for Catalog Storefront Application

  • provides Read/Write API through gRPC

Catalog Storefront Application Responsibilities

  • provide Read API for specific attributes, scope
  • provide Write API for specific attributes, scope
  • store data in an efficient way in own database(s)

Service repository Dependencies

3rd-party dependencies (composer packages)

  • google/protobuf
  • grpc/grpc
  • spiral/php-grpc
  • nette/php-generator

Contributing

Contributions are welcomed! Read the Contributing Guide for more information.

Licensing

This project is licensed under the OSL-3.0 License. See LICENSE for more information.

SF APP Project Documents

catalog-storefront's People

Contributors

aleksejsbaranovs avatar bricht avatar buskamuza avatar duhon avatar kokoc avatar le0n4ik avatar ledian-hymetllari avatar lykhachov avatar melnikovi avatar mslabko avatar owlsmage avatar paliarush avatar ruslankostiv1 avatar serhii-litvynenko-at-rb-com avatar skovalenk avatar vgoncharenko avatar vzabaznov avatar yaroslavgoncharuk avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

catalog-storefront's Issues

Need richer Categories data

Currently, the product feed only sends category paths on each product.

For example, the category in product is missing information about:

  • id
  • name
  • full url to category page
  • parentIds
  • path
  • path by ids
  • others? children_count, description, display_mode, meta_description, meta_title

Search needs richer category attributes. 

Category attributes metadata? Don't know if search needs it. What are they?

Scenario

  • User create or update a category
  • Category updates exported using category feed to a service configured to receive the feed

Notes

  • Depends on #25
  • Reuse categories cache in database
  • Add feed that export categories from cache
  • Feed data should be visible in the service configured to receive this data
     

[17] Read API :: Integration tests :: Wishlist.Model

In the scope of delivery #17 need to fix integration tests Magento.Wishlist.Model*:

Needed branches:

Dev notes:

  • some classes in catalog-storefront repo are autogenerated (marked as "Generated by the Magento PHP proto generator. DO NOT EDIT!") If you still need to edit them to fix test, please mention it in the comment

Example of error:
Stacktrace
Magento\Wishlist\Model\ResourceModel\Item\CollectionTest::testLoadedProductAttributes
Magento\Framework\Exception\LocalizedException: Cannot specify product.

/var/www/html/app/code/Magento/Wishlist/Model/Item.php:376
/var/www/html/dev/tests/integration/tmp/sandbox--23a8fccb4879ec0a108e98befda0d725946cbdfcadd17dfdeeae6767d7a8a6ec/generated/code/Magento/Wishlist/Model/Item/Interceptor.php:95
/var/www/html/dev/tests/integration/testsuite/Magento/Wishlist/Model/ResourceModel/Item/CollectionTest.php:60

Read API :: APIs to read data from catalog storefront data storage

As a Magento Developer, 

I would like to create catalog storefront APIs to read product system and custom attributes data in storefront elasticsearch index

So that

GraphQL can call this API to read the product data via an optimized API. 

Acceptance Criteria

  • APIs are REST & gRPC compatible.
  • APIs can scale up to 1200 concurrent reads with a p90 of <250ms. 
  • APIs must pass all existing Magento GraphQL integration tests.

cc: [~imelnikov] [~iminiailo]

Message broker syncs bundle product system attributes from PIM

Acceptance Criteria

  • APIs are REST compatible.
  • APIs must pass all integration tests.

Scenarios

Scenario 1
Admin creates a new bundle product with options in the back office
Product and options data is available on the catalog storefront

Scenario 2
Admin removes option value of an existing bundle product
Update is available on catalog storefront

Scenario 3
Admin removes option of an existing bundle product
Update is available on catalog storefront

Notes

  • Relations should be represented as options, see #88 for details
  • custom_options should be renamed to options
{
    "id": "2",
    "name": "bundle product",
    "options": [
        {
            "type": "super", // Static value
            "rendererType": "dropdown", // Type of bundle option
            "id": "1",
            "label": "Bundle option",
            "sort_order": "1",
            "values": [
                {
                    "id": "1", // catalog_product_entity_int.value - this is id Variant refers to
                    "label": "configurable-blue",
                    "sort_order": "1",
                    "is_default": false
                },
                {
                    "id": "2", // catalog_product_entity_int.value - this is id Variant refers to
                    "label": "configurable-yellow",
                    "sort_order": "2",
                    "is_default": true
                }
            ]
        }
    ]
},

More in documentation.

Backoffice APIs :: Message broker syncs default product attributes from PIM

As a Magento Developer, 

I would like to create back-office APIs to read product system and custom attributes data  

So that

the data can be returned to the storefront application in an efficient manner.

+Acceptance Criteria+

  • APIs are REST compatible.
  • APIs can scale up to 1200 concurrent reads with a p90 of <250ms. 
  • APIs must pass all integration tests.
    ** Scenario 1 - Create Product - Admin creates a new product in back office and the product data is available in catalog storefront in < 3 seconds (90% cases).
    ** Scenario 2 - Update Product - Admin updates an existing product and the update is available in catalog storefront in <3 secs.
    ** Scenario 3 - Deactivate Product - Admin deactivate an existing product and the product is no longer available in catalog storefront application.

Scenario

User creates or updates product

Default product fields (id, sku, name, meta*, description*, tax_class_id, url_key, visibility, weight) propagated to storefront using new back office API

 

[17] Read API :: Static tests

In the scope of delivery #17 need to fix static tests

http://10.234.203.225:8080/job/Static-Tests/2899
Almost all tests require fixes in proto-generate repository

Needed branches:

Dev notes:

  • some classes in catalog-storefront repo are autogenerated (marked as "Generated by the Magento PHP proto generator. DO NOT EDIT!") If you still need to edit them to fix test, please mention it in the comment

B2B Support: Cat Ext sends category permission data

As a Magento merchant, I want product Recs to support category permissions and shared catalogs (B2B) so that I can show relevant product Recs to all my customers, without assuming everyone can see all products. 

The scope of this work is for the catalog extension to send the required data and for ES to index it. The rest will be covered in epic DATA-1768

Acceptance Criteria 

  • Catalog extension sends sufficient category permissions data 
  • Category permissions data is indexed in ES 

Order management receives sku updates from back office API

Order Management should be informed about actual SKUs available in the back-office so that it can validate placed orders. Having that, every time product SKU is added/updated/deleted in BackOffice, order management should receive this information.

Scenario
The merchant creates/changes product
Order management receives SKU update

[17] Read API :: Integration tests :: Magento.CatalogStorefront

In the scope of delivery #17 need to fix integration tests Magento.CatalogStorefront*:

Needed branches:

Dev notes:

  • some classes in catalog-storefront repo are autogenerated (marked as "Generated by the Magento PHP proto generator. DO NOT EDIT!") If you still need to edit them to fix test, please mention it in the comment

List of tests:
Magento.CatalogStorefront.Model.Storage.ClientAdapterTest.testBulkInsert
Magento.CatalogStorefront.Model.Storage.ClientAdapterTest.testBulkInsertWithWrongMappingType
Magento.CatalogStorefront.Model.Storage.ClientAdapterTest.testNotFoundIndex
Magento.CatalogStorefront.Model.Storage.ClientAdapterTest.testNotFoundItem
Magento.CatalogStorefront.Model.Storage.ClientAdapterTest.testStorageException
Magento.CatalogStorefront.Model.Storage.ClientAdapterTest.testBulkInsert
Magento.CatalogStorefront.Model.Storage.ClientAdapterTest.testBulkInsertWithWrongMappingType
Magento.CatalogStorefront.Model.Storage.ClientAdapterTest.testNotFoundIndex
Magento.CatalogStorefront.Model.Storage.ClientAdapterTest.testNotFoundItem
Magento.CatalogStorefront.Model.Storage.ClientAdapterTest.testStorageException

Example of error:
Stacktrace
Magento\CatalogStorefront\Model\Storage\ClientAdapterTest::testBulkInsert
Magento\Framework\Exception\CouldNotSaveException: Error occurred while saving 'product' entity in the 'catalog_storefront_v1_scope' index.

/var/www/html/app/code/Magento/CatalogStorefront/Model/Storage/Client/ElasticsearchDataDefinitionAdapter.php:118
/var/www/html/dev/tests/integration/testsuite/Magento/CatalogStorefront/Model/Storage/ClientAdapterTest.php:86

Configurable product is missing its variants data

Looking at Capezio and CatBirds production catalog data that are received by our Catalog service, it was discovered that many of the configurable products are missing its Variants data. 

Without Variants data in the configurable product, our Search POC project is unable to properly find information about variants. 

Acceptance Criteria

  • Investigate why the configurable product is missing its variant data.
  • Provide recommendations on how to fix the issue.

 For example:

This simple product has 829W as its parent (configurable product).

{{ "id": "T0RJNVZ5MURRVWt4TVUwX2RYTV9OakZoTkRneU1UWXROalE0TkMwMFlUTm1MV0kwTXpBdFlUQXlPVEZoTm1aak1EWTI", "mageId": "MAG002247678", "instanceId": "cf1f41bf-eea8-45b0-bffa-500209f8e804", "websiteCode": "us_website", "storeCode": "us_store", "storeViewCode": "us", "sku": "829W-CAI11M", "type": "simple", "lastModifiedTs": "2020-03-16T13:32:01Z", "lastExportTs": "2020-03-16T13:32:03.062Z", "lastPublishedTs": "2020-03-16T13:32:03.062Z", "lastUpdatedTs": "2020-03-01T23:54:48Z", "priceHash": "622507ed", "oosHash": "8a3bb795", "contentHash": "4ea0aea1", "productHash": "d9fb82ce", "name": "2.5\" Charlotte Character Shoe", "description": "<h2><strong>Italian Handcrafted Y-Strap Charlotte Character Shoe </strong></h2><br /><p>Fan kick like a pro in the Charlotte Character Shoe. Features a non-slip heel with leather top lift that creates extra stability and a larger strike zone. Special designed Y-Strap provides a wide range of motion. Slim buckle closure eliminates the worry of the character shoe slipping off during an intense performance.</p><br /><h3><strong>Product Features:</strong></h3><br /><ul><br /><li>Full grain Nappa leather upper</li><br /><li>Suede sole</li><br /><li>Cushioned lining molds to foot</li><br /><li>Leather buckle</li><br /><li>Rounded toe box</li><br /><li>Modern Y strap</li><br /><li>Contoured Broadway style 2.5\" heel with leather top lift</li><br /><li>Begin with street shoe size</li><br /></ul>", "shortDescription": null, "urlKey": "", "url": "https://www.capezio.com/charlotte-25-charlotte-chai-11m-1", "metaDescription": null, "metaKeyword": null, "metaTitle": null, "linksExist": false, "linksTitle": null, "linksPurchasedSeparately": false, "taxClassId": "Taxable Goods", "weight": 1.4, "weightType": null, "image": { "url": "https://www.capezio.com/pub/media/catalog/product/cache/f71c5f67374cf93bf50f823a37fc2a3f/c/a/capezio_25_charlotte_character_shoe_chai_829w.jpg", "label": null }, "smallImage": { "url": "https://www.capezio.com/pub/media/catalog/product/cache/2a5aa275c6c2e762ebaebdada76a24e9/c/a/capezio_25_charlotte_character_shoe_chai_829w.jpg", "label": null }, "thumbnailImage": { "url": "https://www.capezio.com/pub/media/catalog/product/cache/eafa19e0e91a04c9629f7ad085f13b5a/c/a/capezio_25_charlotte_character_shoe_chai_829w.jpg", "label": null }, "swatchImage": null, "mediaGallery": [], "attributes": [ { "name": "amazon_3p", "values": [ "No" ] }, { "name": "asin1", "values": [ "B0764LR5XN" ] }, { "name": "attr_3p", "values": [ "Animal Hot Pink" ] }, { "name": "automation_category", "values": [ "Women Character Shoes" ] }, { "name": "color", "values": [ "Tan" ] }, { "name": "ebay_3p", "values": [ "Yes" ] }, { "name": "eu_shoe_size", "values": [ "42.5" ] }, { "name": "shoes_color", "values": [ "Chai" ] }, { "name": "shoes_size", "values": [ "11" ] }, { "name": "shoes_width", "values": [ "M" ] }, { "name": "uk_shoe_size", "values": [ "9" ] }, { "name": "upc", "values": [ "886275244581" ] }, { "name": "wallmart_3p", "values": [ "Yes" ] }, { "name": "walmart_product_status", "values": [ "Not-Uploaded" ] }, { "name": "walmart_product_validation", "values": [ "Not-Validated" ] }, { "name": "walmart_profile_id", "values": [ "Unassigned" ] }, { "name": "wholesale_price", "values": [ "97.250000" ] }, { "name": "x3_sku", "values": [ "829W-CAI11M" ] } ], "visibility": "Not Visible Individually", "displayable": false, "buyable": true, "inStock": true, "lowStock": false, "categories": [], "currency": "USD", "prices": null, "parents": [ { "id": "T0RJNVZ3X2RYTV9OakZoTkRneU1UWXROalE0TkMwMFlUTm1MV0kwTXpBdFlUQXlPVEZoTm1aak1EWTI", "sku": "829W", "type": "configurable" } ], "options": [], "variants": [], "inventoryHash": "8a3bb795", "deleted": false, "weightUnit": "lbs", "environmentId": "61a48216-6484-4a3f-b430-a0291a6fc066", "createdTs": "2018-05-29T18:22:54Z" }}

{code}

However, looking at 829W, the variant data is empty.
 
{{ "id": "T0RJNVZ3X2RYTV9OakZoTkRneU1UWXROalE0TkMwMFlUTm1MV0kwTXpBdFlUQXlPVEZoTm1aak1EWTI", "mageId": "MAG002247678", "instanceId": "cf1f41bf-eea8-45b0-bffa-500209f8e804", "websiteCode": "us_website", "storeCode": "us_store", "storeViewCode": "us", "sku": "829W", "type": "configurable", "lastModifiedTs": "2020-03-17T09:37:37Z", "lastExportTs": "2020-03-17T09:38:04.620Z", "lastPublishedTs": "2020-03-17T09:38:04.620Z", "lastUpdatedTs": "2020-03-05T19:15:52Z", "priceHash": "622507ed", "oosHash": "8a3bb795", "contentHash": "1a053541", "productHash": "da42323d", "name": "2.5\" Charlotte Character Shoe", "description": "<h2><strong>Italian Handcrafted Y-Strap Charlotte Character Shoe </strong></h2>\r\n<p>Fan kick like a pro in the Charlotte Character Shoe. Features a non-slip heel with leather top lift that creates extra stability and a larger strike zone. Special designed Y-Strap provides a wide range of motion. Slim buckle closure eliminates the worry of the character shoe slipping off during an intense performance.</p>\r\n<h3><strong>Product Features:</strong></h3>\r\n<ul>\r\n<li>Full grain Nappa leather upper</li>\r\n<li>Suede sole</li>\r\n<li>Cushioned lining molds to foot</li>\r\n<li>Leather buckle</li>\r\n<li>Rounded toe box</li>\r\n<li>Modern Y strap</li>\r\n<li>Contoured, Broadway style 2.5\" heel with leather top lift</li>\r\n<li>Begin with street shoe size</li>\r\n</ul>", "shortDescription": null, "urlKey": "charlotte-25-charlotte", "url": "https://www.capezio.com/charlotte-25-charlotte", "metaDescription": "Capezio® Charlotte Character Shoe | Style #: 829W | Special designed Y-Strap pump provides a wide range of motion. Features a non-slip heel with leather top lift that creates extra stability and a larger strike zone. ", "metaKeyword": null, "metaTitle": "Italian Handcrafted Y-Strap Charlotte Character Shoe | Capezio®\r\n\r\n", "linksExist": false, "linksTitle": null, "linksPurchasedSeparately": false, "taxClassId": "PC040144", "weight": null, "weightType": null, "image": { "url": "https://www.capezio.com/pub/media/catalog/product/cache/f71c5f67374cf93bf50f823a37fc2a3f/c/a/capezio_25_charlotte_character_shoe_black_829w.jpg", "label": null }, "smallImage": { "url": "https://www.capezio.com/pub/media/catalog/product/cache/2a5aa275c6c2e762ebaebdada76a24e9/c/a/capezio_25_charlotte_character_shoe_black_829w.jpg", "label": null }, "thumbnailImage": { "url": "https://www.capezio.com/pub/media/catalog/product/cache/eafa19e0e91a04c9629f7ad085f13b5a/c/a/capezio_25_charlotte_character_shoe_black_829w.jpg", "label": null }, "swatchImage": null, "mediaGallery": [], "attributes": [ { "name": "age_group", "values": [ "Adult" ] }, { "name": "avalara_tax_id", "values": [ "PC040144" ] }, { "name": "condition", "values": [ "New" ] }, { "name": "gender", "values": [ "Female" ] }, { "name": "google_category", "values": [ "Apparel & Accessories > Shoes > Athletic Shoes & Sneakers > Athletic Shoes > Ballet & Dance Shoes" ] }, { "name": "replenishable", "values": [ "yes" ] }, { "name": "shoes_heel_height", "values": [ "2.5\"" ] }, { "name": "shoes_levels", "values": [ "Intermediate-Advanced" ] }, { "name": "shoes_material", "values": [ "Nappa Leather" ] }, { "name": "shoes_sole_type", "values": [ "Suede Sole" ] }, { "name": "size_guide", "values": [ "0" ] }, { "name": "walmart_product_status", "values": [ "Not-Uploaded" ] }, { "name": "walmart_product_validation", "values": [ "Not-Validated" ] }, { "name": "walmart_profile_id", "values": [ "Unassigned" ] }, { "name": "wholesale_price", "values": [ "97.250000" ] } ], "visibility": "Catalog, Search", "displayable": true, "buyable": true, "inStock": true, "lowStock": false, "categories": [ "capezio-com/women/shoes/character-shoes", "capezio-com/gift-guide/for-her/over-40", "capezio-com/gift-guide/for-her/over-40-eu", "capezio-com/gift-guide/for-her/over-50", "capezio-com/tap-shoes" ], "currency": "USD", "prices": null, "parents": [], "options": [ { "id": "eyJ0eXBlIjoiY29uZmlndXJhYmxlIiwiaWQiOiIxNjMwNiJ9", "type": "configurable", "name": "Color", "values": [ { "id": "eyJ0eXBlIjoiY29uZmlndXJhYmxlIiwiaWQiOiIyNjUwNzEifQ==", "value": "Black", "minimumPrice": null }, { "id": "eyJ0eXBlIjoiY29uZmlndXJhYmxlIiwiaWQiOiIyNjUwOTcifQ==", "value": "Chai", "minimumPrice": null } ], "allowMulti": false }, { "id": "eyJ0eXBlIjoiY29uZmlndXJhYmxlIiwiaWQiOiIxNjMwNyJ9", "type": "configurable", "name": "Size", "values": [ { "id": "eyJ0eXBlIjoiY29uZmlndXJhYmxlIiwiaWQiOiIyNjY3MjMifQ==", "value": "5.5", "minimumPrice": null }, { "id": "eyJ0eXBlIjoiY29uZmlndXJhYmxlIiwiaWQiOiIyNjY3MjcifQ==", "value": "6", "minimumPrice": null }, { "id": "eyJ0eXBlIjoiY29uZmlndXJhYmxlIiwiaWQiOiIyNjY3MzEifQ==", "value": "6.5", "minimumPrice": null }, { "id": "eyJ0eXBlIjoiY29uZmlndXJhYmxlIiwiaWQiOiIyNjY3MzUifQ==", "value": "7", "minimumPrice": null }, { "id": "eyJ0eXBlIjoiY29uZmlndXJhYmxlIiwiaWQiOiIyNjY3MzkifQ==", "value": "7.5", "minimumPrice": null }, { "id": "eyJ0eXBlIjoiY29uZmlndXJhYmxlIiwiaWQiOiIyNjY3NDMifQ==", "value": "8", "minimumPrice": null }, { "id": "eyJ0eXBlIjoiY29uZmlndXJhYmxlIiwiaWQiOiIyNjY3NDcifQ==", "value": "8.5", "minimumPrice": null }, { "id": "eyJ0eXBlIjoiY29uZmlndXJhYmxlIiwiaWQiOiIyNjY3NTEifQ==", "value": "9", "minimumPrice": null }, { "id": "eyJ0eXBlIjoiY29uZmlndXJhYmxlIiwiaWQiOiIyNjY3NTUifQ==", "value": "9.5", "minimumPrice": null }, { "id": "eyJ0eXBlIjoiY29uZmlndXJhYmxlIiwiaWQiOiIyNjY3NTkifQ==", "value": "10", "minimumPrice": null }, { "id": "eyJ0eXBlIjoiY29uZmlndXJhYmxlIiwiaWQiOiIyNjY3NjMifQ==", "value": "10.5", "minimumPrice": null }, { "id": "eyJ0eXBlIjoiY29uZmlndXJhYmxlIiwiaWQiOiIyNjY3NjcifQ==", "value": "11", "minimumPrice": null } ], "allowMulti": false }, { "id": "eyJ0eXBlIjoiY29uZmlndXJhYmxlIiwiaWQiOiIxNjMwOCJ9", "type": "configurable", "name": "Width", "values": [ { "id": "eyJ0eXBlIjoiY29uZmlndXJhYmxlIiwiaWQiOiIyODc3MTUifQ==", "value": "M", "minimumPrice": null } ], "allowMulti": false } ], "variants": [], "inventoryHash": "8a3bb795", "deleted": false, "weightUnit": null, "environmentId": "61a48216-6484-4a3f-b430-a0291a6fc066", "createdTs": "2018-05-29T18:22:54Z", "attributes.variants": [], "rating": 3.6 }}

Notes

  • After #27 is resolved, need to make sure that variants being added to the feed.
     

Add test for product attributes in export API

Subtask of #28.

We added product EAV attributes to export API. Add tests to validate that product attributes returned correctly. Need to test all types of product attributes.

A new test can be added here dev/tests/api-functional/testsuite/Magento/CatalogExport/ExportTest.php.

The following branches contain changes for new API

Should work on 2.3.4

Message broker syncs product custom options and variants from PIM

Acceptance Criteria

  • All changes should be covered by integration tests.
  • New schema should be sufficient to complete "Add Product To Cart" request

What To Demo

  • The product can be successfully added to the Cart with data received from new schema

Scenarios
Merchant create product variation
Product variation propagated to storefront using new API

Notes

New endpoint that receive product ids and a cursor (id of last received product or 0, in case of first request)
/V1/catalog-export/product/variants?ids[0]=100&ids[0]=10

VariantInterface

/**
  * @return string
  */
public getId();
 
/**
  * @return int[]
  */
public getOptionValueIds();
 
/**
  * @return array
  */
public getPrice();

ProductOptionInterface

/**
  * @return string
  */
public getId();

/**
  * @return string
  */
public getLabel();

/**
  * @return int
  */
public getSortOrder();

/**
  * @return bool
  */
public getRequired();

/**
  * @return bool
  */
public getRenderType();

/**
  * @return ProductOptionValue[]
  */
public getProductOptionValue();

ProductOptionValueInterface

/**
  * @return string
  */
public getId();

/**
  * @return string
  */
public getLabel();

/**
  * @return int
  */
public getSortOrder();

/**
  * @return bool
  */
public getDefault();

/**
  * @return bool
  */
public getImageUrl();

Price calculation details:
We are going to store a price as plain float value. It mens that in terms of Custom options with price in % for option we are going to precalculate the price for Variant by the next formula:

Variant::price = [
      [
             'regular_price' => Product::regular_price_for_customer_group_1 * Product::option[k]::option_value[p]::price/100,
             'final_price' = Product::final_price_for_customer_group_1 * Product::option[k]::option_value[p]::price/100,
             'scope' => customer_group_1
      ],
      [
             'regular_price' => Product::regular_price_for_customer_group_2 * Product::option[k]::option_value[p]::price/100,
             'final_price' = Product::final_price_for_customer_group_2 * Product::option[k]::option_value[p]::price/100,
             'scope' => customer_group_2
      ],
      ...
]

For custom options with fixed price:

Variant::price = [
      [
             'regular_price' => Product::option[k]::option_value[p]::regular_price/100
             'final_price' = Product::option[k]::option_value[p]::final_price/100,
             'scope' => customer_group_1
      ],
      [
             'regular_price' => Product::option[k]::option_value[p]::regular_price/100
             'final_price' = Product::option[k]::option_value[p]::final_price/100,
             'scope' => customer_group_2
      ],
      // yes, it repeats the same price value for each scope, as a temp solution it should be fine.
]

Where: Product::price - price of the Product itself for each customer group respectively, and we are going to use the value from product_price_index; k - index of option, p - index of each items in option
By that logic we are going to end up with matrix of Variants that represents each option value (let's be m) for each Selected Custom option (let's be n). The overall count of Variant has to be m*n.

Calculation of price MUST take into account:

  1. Spacial price
  2. Group price
  3. Tier price
  4. Catalog rule

Event types for catalog storefront API

Overview

Principal scheme of updates coming to Catalog Storefront:
Changes in Admin Panel -> Magento -> Queue -> Catalog Storefront

Currently, Magento emits only one type of events which mean two operations update product data and delete product. The product is deleted in case of empty payload in the message. This approach is unreliable because failures to collect data on the emitter side (Magento) will lead to product delete operations on the storefront.

Please introduce explicit operation types in communications.

Suggested event types: product_updated, product_created, product_deleted, category_updated, category_created, category_deleted

AC

  • Events contain an explicitly defined type of operations (create, update, delete)
  • Catalog storefront deletes entities only on delete events
  • Product and category entities are using this approach

Variant data under Configurable product is missing Image Url

Variant data under the Configurable product is missing Image Url.

In some situations, the Variant data is missing altogether - https://jira.corp.magento.com/browse/SEARCH-134 from the configurable product, however, this is a separate issue.

In this case, even if Variant data is available, we do not have the image URL of the variant data.

This is needed if we want the ability to show the variant image if the query term matches the variant of the configurable product, e.g. searching for "pants women red" on Luma should show us women red pants.

Notes

  • After #19 is complete, need to make sure feed contains product image URLs
     

Storefront is compatible with 2.4

Ensure catalog storefront code is compatible with 2.4 develop (eventually GA)

Acceptance Criteria

  • Create WebAPI build on CI with 2.4-develop branches and storefront (ce and ee branches)
  • WebAPI tests green on CI with PHP 7.4
  • Catalog storefront repository is compatible with PHP 7.4 and PHPUnit 9

Repos where need to make change

  • [https://github.com/magento/magento2ce]
  • [https://github.com/magento/magento2ee|https://github.com/magento/magento2ce]
  • [https://github.com/magento/catalog-storefront-ce]
  • [https://github.com/magento/catalog-storefront-ee|https://github.com/magento/catalog-storefront-ce]
  • [https://github.com/magento/saas-export] (if ECP-513 merged by this point)

Message broker syncs product prices from PIM

As a Magento Developer, 

I would like to create back-office APIs to read product pricing data  

So that

the data can be returned to the storefront application in an efficient manner.

Acceptance Criteria

  • APIs are REST compatible.
  • APIs must pass all integration tests.

Scenario 1 - Create Product - Admin creates a new product price in the back office and the product data is available in catalog storefront in < 3 seconds (90% cases).
Scenario 2 - Update Product - Admin updates an existing product and the update is available in catalog storefront in <3 secs.
Scenario 3 - Deactivate Product - Admin deactivate an existing product and the product is no longer available in catalog storefront application.

 

cc: [~imelnikov] [~iminiailo]

Staging support

As a merchant, I want to have a staging functionality working

AC:

  • Staging tests overrides(ee/dev/tests/api-functional/testsuite/GraphQl/Staging) which skip tests related to non-working functionality are eliminated
  • All staging webapi tests are green
  • Runtime calls from catalog storefront to monolith are eliminated
  • Staging functionality works as a separate service (possibility to deploy modules separately)
  • Preview requests are handled by staging service/modules

Design references

Message broker syncs product reviews from PIM

As a Magento Developer, 

I would like to create back-office APIs to read product review data  

So that

the data can be returned to the storefront application in an efficient manner.

Acceptance Criteria

  • APIs are REST compatible.
  • APIs must pass all integration tests.

Scenario

  • A user creates or updates product review
  • Product review changes propagated to storefront using new back office API

 Implementation notes: #18 (comment)

 

[17] Read API :: Integration tests :: Magento.CatalogStaging

In the scope of delivery #17 need to fix integration tests Magento.CatalogStaging.Model.CategoryTreeTest.testCategoryTreeReturnsAllChildCategories

Needed branches:

Dev notes:

  • some classes in catalog-storefront repo are autogenerated (marked as "Generated by the Magento PHP proto generator. DO NOT EDIT!") If you still need to edit them to fix test, please mention it in the comment

Magento.CatalogStaging.Model.CategoryTreeTest.testCategoryTreeReturnsAllChildCategories (from Magento_CatalogStaging_Model_CategoryTreeTest)

Failing for the past 1 build (Since Failed#4745 )
Took 3.8 sec.
Stacktrace
Magento\CatalogStaging\Model\CategoryTreeTest::testCategoryTreeReturnsAllChildCategories
Failed asserting that an array does not have the key 'errors'.

/var/www/html/dev/tests/integration/testsuite/Magento/CatalogStaging/Model/CategoryTreeTest.php:174

[17] Read API :: Integration tests :: Wishlist.Controller

In the scope of delivery #17 need to fix integration tests Magento.Wishlist.Controller*:

Needed branches:

Dev notes:

  • some classes in catalog-storefront repo are autogenerated (marked as "Generated by the Magento PHP proto generator. DO NOT EDIT!") If you still need to edit them to fix test, please mention it in the comment

Example of error:
Stacktrace
Magento\Wishlist\Controller\Index\IndexTest::testItemColumnBlock
Failed asserting that 0 matches expected 1.

/var/www/html/dev/tests/integration/testsuite/Magento/Wishlist/Controller/Index/IndexTest.php:70

B2B Support: Cat Ext sends customer-specific pricing

As a Magento merchant, I need displayed Recommendations to correctly reflect accurate customer-specific pricing for all my customer groups.

Ex, if I have defined specific pricing for my wholesale customer segment, then a logged-in customer in that segment should see correct product pricing reflected in the Recs widgets shown to them on the site.  

The work under this epic is scoped to the Catalog extension through the ES index only. The rest of the work will be covered in epic DATA-1769

Acceptance Criteria

  • Customer group pricing is included in the catalog export
  • Customer group pricing is indexed in ES with rest of catalog data

 Technical questions: 

  • how to store this data in the index efficiently so that it was being used to filter results at runtime (ex: when fetching recommendations or returning search results)

 

 

[17] Read API :: Integration tests :: Customer*

PR

Fixed in magento/magento2#28746

In the scope of delivery #17 need to fix integration tests Magento.Customer*:

Needed branches:

Dev notes:

  • some classes in catalog-storefront repo are autogenerated (marked as "Generated by the Magento PHP proto generator. DO NOT EDIT!") If you still need to edit them to fix test, please mention it in the comment

Example of error:
Stacktrace
Magento\Customer\Block\Adminhtml\Edit\Tab\View\WishlistTest::testWishListGrid
Failed asserting that actual size 0 matches expected size 1.

Magento.Customer.Block.Adminhtml.Edit.Tab.View.WishlistTest.testWishListGrid
Magento.Customer.Block.Adminhtml.Edit.Tab.View.WishlistTest.testWishListGrid
Magento.Customer.Controller.Section.LoadTest.testWishListCounterUseQty
Magento.Customer.Controller.Section.LoadTest.testWishListCounterNotUseQty
Magento.Customer.Controller.Section.LoadTest.testWishListCounterUseQty
Magento.Customer.Controller.Section.LoadTest.testWishListCounterNotUseQty

[17] Read API :: Integration tests :: ConfigurableProduct.*

In the scope of delivery #17 need to fix integration tests Magento.ConfigurableProduct*:

Needed branches:

Dev notes:

  • some classes in catalog-storefront repo are autogenerated (marked as "Generated by the Magento PHP proto generator. DO NOT EDIT!") If you still need to edit them to fix test, please mention it in the comment

Example of error:
Stacktrace
Magento\ConfigurableProduct\Block\Product\View\Type\ConfigurableProductPriceTest::testConfigurablePrice
Failed asserting that '

As low as $0.00
' matches PCRE pattern "/As low as</span>.*<span.data-price-amount="10".$10.00</span></span>/".

/var/www/html/dev/tests/integration/testsuite/Magento/ConfigurableProduct/Block/Product/View/Type/ConfigurableProductPriceTest.php:193
/var/www/html/dev/tests/integration/testsuite/Magento/ConfigurableProduct/Block/Product/View/Type/ConfigurableProductPriceTest.php:81

Message broker syncs product extension attributes from PIM

As a Magento Developer, 

I would like to create back-office APIs to read product extension attributes data  

So that

the data can be returned to the storefront application in an efficient manner.

Acceptance Criteria

  • APIs are REST compatible.
  • APIs must pass all integration tests.

Scenario

  • A user creates, updates or deletes product extension attributes
  • Product extension attributes changes propagated to storefront using new back office API
     

 Implementation notes: #18 (comment)

[#25 - Parent] Message broker :: Categories :: Sync category data from back-office/PIM

As a Magento Developer, 

I would like to create back-office APIs to read category data  

So that

the data can be returned to the storefront application in an efficient manner.

Acceptance Criteria

  • APIs are REST compatible.
  • APIs must pass all integration tests.
  • The scope of this ticket covers all the category attributes that can be exposed via GraphQL.

Scenario

  • A user creates, updates or deletes a category
  • Category changes propagated to storefront using new back office API

Notes

  • Create configuration to save categories in the cache table
  • Expose endpoint for category data in the export API
  • Make sure updates propagated to GraphQl
  • #18 (comment)

https://wiki.corp.magento.com/display/ARCH/Categories+Export
https://wiki.corp.magento.com/display/ARCH/Product+Prices+Export

Message broker syncs downloadable product system attributes from PIM

Acceptance Criteria

  • APIs are REST compatible.
  • APIs must pass all integration tests.

Scenarios
Scenario 1

  • Admin creates new downloadable product with links in the back office
  • Product with links is available on the catalog storefront

Scenario 2

  • Admin removes link of downloadable product in the back office
  • Removed link is not available on the catalog storefront

Notes

  • Relations should be represented as options, see #88 for details
  • custom_options should be renamed to options
  • As new API does not allow to receive all data for downloadable product, we can't remove old data provider

proto-schema is:

message Sample {
    string url = 1;
    string label = 2;
    int32 sort_order = 3;
}

GraphQL schema:

type DownloadableProductSamples {
    id: 
    title:
    sort_order: 
    sample_url: 
    sample_type: @deprecated
    sample_file: @deprecated
}

More in documentation.

Developer onboarding

Documentation for developer onboarding that includes:

  • sources of repos and branches to be used
  • steps for composing the project
  • tools to run the application locally with steps and common scenarios

Backoffice APIs :: Message broker syncs product custom options from PIM

As a Magento Developer, 

I would like to create back-office APIs to read product custom options data  

So that

the data can be returned to the storefront application in an efficient manner.

Acceptance Criteria

  • APIs are REST compatible.
  • APIs must pass all integration tests.

Scenario 1 - Create Product - Admin creates a new Product custom option in the back office and the product data is available in catalog storefront in < 3 seconds (90% cases).
Scenario 2 - Update Product - Admin updates an existing Product custom option and the update is available in catalog storefront in <3 secs.
Scenario 3 - Deactivate Product - Admin deactivate an existing Product custom option and the Product custom option is no longer available in catalog storefront application.

Message broker syncs product images and videos from PIM

Scenario
1

  • User opens edit product page, creates, updates or delete product images and saves the product
  • Product images changes propagated to storefront using new back office API and the user can see changes in the GraphQl

2

  • User opens edit product page, creates, updates or delete product video and saves the product
  • Product video changes propagated to storefront using new back office API and the user can see changes in the GraphQl

Notes
The following fields need to be added to the back-office API. URL should be the original image URL (full URL of the image added by the admin for a particular store view). URLs should not be read from the cache.

ProductInterface

    media_gallery

ItemInterface

    url // for video URL of the preview image
    label
    type (base, small, thumbnail, swatch, video)
    video_attributes
    sort_order
    media_type????(Igor to check)

VideoAttributes

    media_type,
    video_provider,
    video_url,
    video_title,
    video_description,
    video_metadata

Changes in GQL schema:

[17] Read API :: Integration tests :: MultipleWishlist.*

In the scope of delivery #17 need to fix integration tests Magento.MultipleWishlist*:

Needed branches:

Dev notes:

  • some classes in catalog-storefront repo are autogenerated (marked as "Generated by the Magento PHP proto generator. DO NOT EDIT!") If you still need to edit them to fix test, please mention it in the comment

Example of error:

Stacktrace
Magento\MultipleWishlist\Block\Customer\Wishlist\Item\Column\ManagementTest::testDisplayWishListManagementBlock
Failed asserting that null is not null.

/var/www/html/dev/tests/integration/testsuite/Magento/MultipleWishlist/Block/Customer/Wishlist/Item/Column/ManagementTest.php:73

Message broker syncs configurable product system attributes from PIM

Acceptance Criteria

  • APIs are REST compatible.
  • APIs must pass all integration tests.

Scenarios

Scenario 1
Admin creates a new configurable product with variations in the back office
Product and variations data is available on the catalog storefront

Scenario 2
Admin removes variation of an existing configurable product
Update is available on catalog storefront

Notes

  • Need to add configurable product relations to child products
  • Relations should be represented as options, see #88 for details
  • custom_options should be renamed to options
     
{
    "id": "1",
    "name": "configurable product",
    "options": [
        {
            "type": "super",
            "id": "1", // catalog_product_super_attribute.product_super_attribute_id
            "label": "Color", // catalog_product_super_attribute_label.value
            "sort_order": "1", // catalog_product_super_attribute.position
            "values": [
                {
                    "id": "1", // catalog_product_entity_int.value - this is id Variant refers to
                    "label": "Red"
                },
                {
                    "id": "2", // catalog_product_entity_int.value - this is id Variant refers to
                    "label": "Blue"
                }
            ]
        }
    ]
},

More in documentation.

storefront:catalog:sync command need to be decoupled from indexer

The purpose of storefront:catalog:sync command is initial product sync up. It is currently on back office side, which means that by this design PIMs will have to implement similar command on their side.

This command probably should be on the message broker side (message broker should be able sync incremental changes by events or all of the changes by some other event of when manually triggered). This probably would be the ideal state. But we could also do interim solution

  • Remove this command and implement it in the message broker in the separate story
  • Remove dependency on indexer and move to message broker in the separate story

[17] Read API :: Integration tests :: Magento.TargetRule

In the scope of delivery #17 need to fix integration tests Magento.TargetRule.Model.RuleTest.testCategoryCondition:

Needed branches:

Dev notes:

  • some classes in catalog-storefront repo are autogenerated (marked as "Generated by the Magento PHP proto generator. DO NOT EDIT!") If you still need to edit them to fix test, please mention it in the comment

Stacktrace
Magento\TargetRule\Model\RuleTest::testCategoryCondition with data set "Product category does not contain 5 AND Product SKU contains "simple_product"" (array(array(3, 4), array(3, 4, 5), array(3, 4, 5)), array(array('Magento\TargetRule\Model\Acti...ombine', 'all', '1'), array('Magento\TargetRule\Model\Acti...ibutes', 'category_ids', '!{}', 5, false, 'constant'), array('Magento\TargetRule\Model\Acti...ibutes', 'sku', '{}', 'simple_product', false, 'constant')), array('simple_product_1'))
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
Array (

  • 0 => 'simple_product_1'
    )

/var/www/html/dev/tests/integration/testsuite/Magento/TargetRule/Model/RuleTest.php:143

Message broker syncs product attributes from PIM

As a Magento Developer, 

I would like to create back-office APIs to read product attributes data  

So that

the data can be returned to the storefront application in an efficient manner.

Acceptance Criteria

  • APIs are REST compatible.
  • APIs must pass all integration tests.

Scenario 1 - Create Product - Admin creates a new product in the back office and the product data is available in catalog storefront in < 3 seconds (90% cases).
Scenario 2 - Update Product - Admin updates an existing product and the update is available in catalog storefront in <3 secs.
Scenario 3 - Deactivate Product - Admin deactivate an existing product and the product is no longer available in catalog storefront application.

 

[17] Read API :: Integration tests ::Magento.VersionsCms

PR

Fixed in magento/magento2#28746

In the scope of delivery #17 need to fix integration tests
Magento.VersionsCms.Model.Hierarchy.NodeTest.testAppendPageToNodes

Needed branches:

Dev notes:

  • some classes in catalog-storefront repo are autogenerated (marked as "Generated by the Magento PHP proto generator. DO NOT EDIT!") If you still need to edit them to fix test, please mention it in the comment

Magento.VersionsCms.Model.Hierarchy.NodeTest.testAppendPageToNodes (from Magento_VersionsCms_Model_Hierarchy_NodeTest)

Failing for the past 1 build (Since Failed#4745 )
Took 4.1 sec.
Stacktrace
Magento\VersionsCms\Model\Hierarchy\NodeTest::testAppendPageToNodes
Session messages do not meet expectations array (
)
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
Array (

  • 0 => 'You saved the page.'
  • 1 => 'You duplicated the page.'
    )

/var/www/html/dev/tests/integration/framework/Magento/TestFramework/TestCase/AbstractController.php:250
/var/www/html/dev/tests/integration/testsuite/Magento/VersionsCms/Model/Hierarchy/NodeTest.php:73

Message broker syncs gift card product with predefined amounts system attributes from PIM

Acceptance Criteria

  • APIs are REST compatible.
  • APIs must pass all integration tests.

Scenarios
Scenario 1

  • Admin creates a new gift card product with predefined amounts in the back office
  • Product with selected predefined amounts is available on the catalog storefront

Notes

  • Relations should be represented as options, see #88 for details
  • custom_options should be renamed to options
{
    "id": "3",
    "name": "gift card product no open amount",
    "options": [
        {
            "type": "giftcard_amount",
            "id": "134", // eav_attribute.attribute_id
            "values": [
                {
                    "id": "1", // magento_giftcard_amount.value_id
                    "price": "10",
                    "label": "10"
                },
                {
                    "id": "2", // magento_giftcard_amount.value_id
                    "price": "10",
                    "label": "10"
                }
            ]
        }
    ]
},

More in documentation.

[17] Read API :: Integration tests :: Magento.StorefrontGraphQl

In the scope of delivery #17 need to fix integration tests Magento.StorefrontGraphQl*:

Needed branches:

List of tests:
Magento.StorefrontGraphQl.Model.Query.ScopeProviderTest.testGetScopesWithoutStore
Magento.StorefrontGraphQl.Model.Query.ScopeProviderTest.testGetScopesWithoutGroupId
Magento.StorefrontGraphQl.Model.Query.ScopeProviderTest.testGetScopesWithoutStore
Magento.StorefrontGraphQl.Model.Query.ScopeProviderTest.testGetScopesWithoutGroupId

Example of error:
Magento.StorefrontGraphQl.Model.Query.ScopeProviderTest.testGetScopesWithoutStore (from Magento_StorefrontGraphQl_Model_Query_ScopeProviderTest)

Failing for the past 1 build (Since Failed#4745 )
Took 9 ms.
Stacktrace
Magento\StorefrontGraphQl\Model\Query\ScopeProviderTest::testGetScopesWithoutStore
Magento\Framework\GraphQl\Exception\GraphQlNoSuchEntityException: There is no store in extension attributes

/var/www/html/app/code/Magento/StorefrontGraphQl/Model/Query/ScopeProvider/StoreScope.php:29
/var/www/html/app/code/Magento/StorefrontGraphQl/Model/Query/ScopeProvider.php:49
/var/www/html/dev/tests/integration/testsuite/Magento/StorefrontGraphQl/Model/Query/ScopeProviderTest.php:107

Dev notes:

  • some classes in catalog-storefront repo are autogenerated (marked as "Generated by the Magento PHP proto generator. DO NOT EDIT!") If you still need to edit them to fix test, please mention it in the comment

Storefront APIs :: APIs to write data to catalog storefront data storage

As a Magento Developer, 

I would like to create catalog storefront APIs to write product & category system and custom attributes data in storefront elasticsearch index

So that

Message Broker can call this API to write products & categories data via an optimized API. 

Acceptance Criteria

  • APIs are REST & gRPC compatible.
  • APIs must pass all existing Magento GraphQL integration tests.

Story branches

Install module dependencies with git installation

Some modules can declare dependencies that will not be installed in case of git installation

E.g.
https://github.com/magento/catalog-storefront/blob/api-proto/app/code/Magento/CatalogStorefrontApi/composer.json#L10

In the future, we plan to have more such dependencies.
Need to create an approach to easily install these dependencies on:

  1. Jenkins builds (DevOps team already has an item in the backlog: https://jira.corp.magento.com/browse/MTS-1275)
  2. Local developments

Message broker syncs product attribute data from PIM

As a Magento Developer, 

I would like to create back-office APIs to read product attributes data  

So that

the data can be returned to the storefront application in an efficient manner.

Acceptance Criteria

  • APIs are REST compatible.
  • APIs must pass all integration tests.

Scenario 1 - Create Product - Admin creates a new product in the back office and the product data (product data, not attribute entity) is available in catalog storefront in < 3 seconds (90% cases).
Scenario 2 - Update Product - Admin updates an existing product and the update is available in catalog storefront in <3 secs. (product data, not attribute entity)
Scenario 3 - Deactivate Product - Admin deactivate an existing product and the product is no longer available in catalog storefront application. (product data, not attribute entity)

 

Message broker syncs category extension attributes from PIM

As a Magento Developer, 

I would like to create back-office APIs to read category extension attributes data  

So that

the data can be returned to the storefront application in an efficient manner.

Acceptance Criteria

  • APIs are REST compatible.
  • APIs must pass all integration tests.

Scenario

  • A user creates, updates or deletes category extension attributes
  • Category extension attributes changes propagated to storefront using new back office API

Implementation notes: #18 (comment)
 

 

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.