Incremental Feed JSON Data Format

This page describes the data format to follow when implementing a JSON integration with Fredhopper.

Files Required

The table below lists all the files that can be expected as part of the incremental update:

Filename	Description
meta.json	This file is not required as changes to the meta cannot be made during an incremental
products-n.json	This file(s) should only contain references to the product ids that are to be added/amended as part of the incremental update (optional).
variants-n.json	Similar in logic to the products json file(s), this file(s) should only contain references to any variant ids that are to be added/amended (optional).

If your data model is such that there is no concept of variants, then the variants json file can be omitted. The products and variants JSON files should only ever contain a maximum of 10,000 items within them in order to be processed successfully. Additional files should be added and the item data should be split out when there are more than 10,000 products and/or 10,000 variants, e.g. products-0.json, products-1.json, products-2.json, variants-0.json, variants-1.json, variants-2.json etc.

Required Files

For all incremental updates, it is highly recommended that the meta file is not included. If the type of 'Operation' is update or delete, you can provide the relevant products-n.json or the variants-n.json files only. If the type of 'Operation' is add or replace, you will need to ensure that all of the required data files are included.

Operation Type

The full re-index process always expects that the entire product catalogue be provided within the data files provided. This differs in comparison to the incremental update process as here we may wish to only update an attribute value, add a new product or delete a variant from the catalogue entirely. The way that we inform Fredhopper which action to perform in an incremental update is defined by the 'operation' property that should be included within the data. For example, if a variant with two locales must be removed it is sufficient to include only one delete operation property in variants-0.json (see in the sample data below how this is done).

Please be aware that you must not specify an attribute_value_id for 'operation_type' entries.

The 'operation_type' property is defined within the products-n.json and (if required) variants-n.json for each product / variant respectively and it can be set to one of four values below. If no "operation_type" property is configured for an item within the file, then the 'add' operation will be applied by default.

Operation Type	Description
update	An existing product or variant item needs to have some of its data modified. By default, if the given item does not exist within the FAS index, no changes will be completed. This default behaviour can be overridden in the system.xml settings so that the item is processed as an add type of update instead. This operation_type cannot be used to nullify an existing attribute value or remove an attribute completely for an item - the replace operation must be used in this case.
replace	An existing product or variant item (and all its attributes) should be removed completely from the FAS index and will be replaced with the new data provided within the incremental update.
add	A new product or variant item has been defined within the incremental update and the attribute values need be loaded for the first time. By default, if the item exists already within the index, FAS will apply the replace logic to the existing item in this case. This default behaviour can be overridden in the system.xml so that the existing item does not get replaced and a warning will be displayed in the logs for the affected item(s) instead.
delete	An existing product or variant item needs to be removed completely from the FAS index.

*An item can be either a product or variant.

FAS will execute all operations on the specified item so, if both a product and its variants need updating, all of those updates must be provided explicitly. The only exception to this rule is that, deleting a product that has variants attached, will remove both the product and all variants automatically without the need for each variant to be deleted explicitly.

Fast Incremental Updates

The speed that it takes to process and load an incremental update is already much faster than it takes a full data upload to complete. However, it is possible to have a faster, more expedited update process when sending an incremental update batch that contains only numerical and/or basic text values. In this situation, some of the standard data processing tasks and verification checks that are built into the process can be skipped because they are not necessary. This method should be used whenever possible to provide near real-time updates of numerical and/or text values, for example to apply stock and/or price updates. For this to work, the operation_type can only be 'update' and the input files will only contain a subset of the product/variant attributes that are to be updated.

The data API will automatically complete a fast incremental update when all of the following conditions have been met:

• The products-n.json and (if required) variants-n.json file(s) is(are) present in data-incremental.zip file

• The operation_type values for all changes are set to 'update' (using any other value will result in a 'normal' incremental update)

• The data type of all the attribute values to be updated are either int, float or text (using any other data type will result in a 'normal' incremental update)

Incremental Product Data - Input Examples

The data model depicted in the examples below represents a single universe, multi locale deployment which has two categories (womens & shoes) that contain two products and three variants in total. The update scenario that we would like to achieve is as follows:

• p1 is updated with a new colour attribute value (blue)

• p2 is added as a new item to the product catalogue

• v1 is removed from the product catalogue

• v2 is updated with a new price value (30.00)

products-0.json

{
    "products": [
        {
            "attributes": [
                {
                    "attribute_id": "colour",
                    "values": [
                        {
                            "locale": "de_DE",
                            "value_id": "blue",
                            "value": "Blauw"
                        },
                        {
                            "locale": "en_GB",
                            "value_id": "blue",
                            "value": "Blue"
                        }
                    ]
                }
            ],
            "operation": "update",
            "product_id": "p1"
        },
        {
            "attributes": [
                {
                    "attribute_id": "categories",
                    "values": [
                        {
                            "value": "shoes"
                        }
                    ]
                },
                {
                    "attribute_id": "colour",
                    "values": [
                        {
                            "locale": "de_DE",
                            "value_id": "red",
                            "value": "Rot"
                        },
                        {
                            "locale": "en_GB",
                            "value_id": "red",
                            "value": "Red"
                        }
                    ]
                },
                {
                    "attribute_id": "stock",
                    "values": [
                        {
                            "value": "10"
                        }
                    ]
                }
            ],
            "operation": "add",
            "product_id": "p2"
        }
    ]
}

variants-0.json

{
    "variants": [
        {
            "variant_id": "v1",
            "operation": "delete"
        },
        {
            "attributes": [
                {
                    "attribute_id": "price",
                    "values": [
                        {
                            "value": "30.00"
                        }
                    ]
                }
            ],
            "variant_id": "v2",
            "operation": "update"
        }
    ]
}