Custom Data Format

Overview

The following section details the standard data input format that is expected by the Upload service. It also explains how these files should be transmitted to Fredhopper's Managed Services environment and for the data to be triggered on the instance. Once the additional data has been successfully uploaded and processed, it will be integrated with the next product feed update (full or incremental). The same custom data will be maintained for all subsequent load-data triggers until a new version of the custom data is uploaded using this process.

Uploading and Triggering the processing of data using Upload

Once the custom file(s) is ready in the predefined format, it then needs to be compressed using the zip archive specification (application/zip) before it can be uploaded. An MD5 checksum of the newly created ZIP file also needs to be generated to verify the file.

Fredhopper offers a REST and File (FTP) API. The File (FTP) API is suitable for development purposes, while for production use we strongly recommend the use of the REST API.

Region	Countries	Data Endpoint
EU	All	https://my.eu1.fredhopperservices.com/upload:[service_instance]/data/input/

The service_instance is either live1, test1, test2 or test3, depending on the environments that are available for you.

Steps

1. Compress the data file(s) into one data.zip file

2. Create MD5 checksum of data.zip file to validate that upload file is not corrupt, e.g.:

DB211B14E25CFB18792C3B3697CD0394 data.zip

1. Upload the data.zip file and the MD5 checksum to environment

REST URL via HTTP-PUT:

https://my.eu1.fredhopperservices.com/upload:[service_instance]/data/input/data.zip?checksum=[checksum]

cURL example:

curl -D - -u "[cloud_username]":"[cloud_password]" -X PUT -H "Content-Type: application/zip" --data-binary @data.zip https://my.eu1.fredhopperservices.com/upload:live1/data/input/data.zip\?checksum=19afc8c99ec7ea27f0dba8e1c4fcff1f

1. Save the data-id value from the response body for tracking purposes

HTTP/1.1 100 Continue
 
HTTP/1.1 201 Created
Date: Fri, 21 Feb 2020 15:31:55 GMT
Server: Jetty(9.4.22.v20191022)
Location: https://my.eu1.fredhopperservices.com/upload:live1/data/input/2020-02-21_15-31-55
Content-Type: text/plain
Content-Length: 28
Connection: close
 
data-id=2020-02-21_15-31-55

1. Trigger the processing of the newly uploaded data by creating a trigger that included the data-id value that was captured in step 4

REST URL:

https://my.eu1.fredhopperservices.com/upload:[service_instance]/trigger/finalize

cURL examples:

curl -D - -u "[cloud_username]":"[cloud_password]" -X PUT -H "Content-Type: text/plain" --data-binary "data-id=[data-id]" https://my.eu1.fredhopperservices.com/upload:[service_instance]/trigger/finalize

curl -D - -u "[cloud_username]":"[cloud_password]" -X PUT -H "Content-Type: text/plain" --data-binary "data-id=2020-02-21_15-31-55" https://my.eu1.fredhopperservices.com/upload:live1/trigger/finalize

Example response (note the Location value in the header):

HTTP/1.1 201 Created
Date: Fri, 21 Feb 2020 15:47:17 GMT
Server: Jetty(9.4.22.v20191022)
Location: https://my.eu1.fredhopperservices.com/upload:live1/trigger/finalize/2020-02-21_15-47-17
Content-Length: 0
Connection: close

1. Monitor the process by using the value of the 'Location' in the previous step and viewing the newly-created status file until the content is either SUCCESS or FAILURE.

REST URL:

https://my.eu1.fredhopperservices.com/upload:[service_instance]/trigger/finalize/[trigger-id]/status

cURL example:

curl -s -D - -u "[cloud_username]":"[cloud_password]" https://my.eu1.fredhopperservices.com/upload:[service_instance]/trigger/finalize/[trigger-id]/status

curl -s -D - -u "[cloud_username]":"[cloud_password]" https://my.eu1.fredhopperservices.com/upload:live1/trigger/finalize/2020-02-21_15-47-17/status

Example response:

HTTP/1.1 200 OK
Date: Fri, 21 Feb 2020 15:49:09 GMT
Server: Jetty(9.4.22.v20191022)
Content-Type: text/plain
Content-Length: 57
Connection: close
 
RUNNING
Started execution at Fri Feb 21 15:47:17 UTC 2020

... and when successfully completed, the response should appear as follows:

HTTP/1.1 200 OK
Date: Fri, 21 Feb 2020 15:51:15 GMT
Server: Jetty(9.4.22.v20191022)
Content-Type: text/plain
Content-Length: 8
Connection: close
 
SUCCESS

Process diagram

Diagram illustrating the steps above