# Custom Data Feed Data Uploads

## Overview <a href="#fulldata-csv-stepone-createziparchive" id="fulldata-csv-stepone-createziparchive"></a>

If you are implementing a custom feed for additional data enrichment you will need to send these to the *upload* service.

The following section details the standard data input format that is expected by the Upload service. It also explains how these files should be transmitted to Fredhopper's Managed Services environment and for the data to be triggered on the instance.

Once the additional data has been successfully uploaded and processed, it will be integrated with the next product feed update (full or incremental). The same custom data will be maintained for all subsequent load-data triggers until a new version of the custom data is uploaded using this process.

{% hint style="info" %}
File Name: `data.zip`

Example URL: `https://my.eu1.fredhopperservices.com/upload`
{% endhint %}

## Upload Steps <a href="#fulldata-csv-stepone-createziparchive" id="fulldata-csv-stepone-createziparchive"></a>

These are the steps to follow when uploading data to Fredhopper, this is the same for Item data, suggest data, and custom uploads.

The file names and URL's to upload too will differ and are covered in the examples.

### 1. Create a ZIP archive <a href="#fulldata-csv-stepone-createziparchive" id="fulldata-csv-stepone-createziparchive"></a>

Create a data.zip file containing the data you need to upload to Fredhopper.

### 2. Create MD5 checksum <a href="#fulldata-csv-steptwo-createmd5checksum" id="fulldata-csv-steptwo-createmd5checksum"></a>

Generate an [md5 checksum](http://linux.die.net/man/1/md5sum) value zip file.

{% code title="Example Request" overflow="wrap" %}

```
md5sum data.zip > data.zip.md5
```

{% endcode %}

This checksum we'll use to validate the upload.

### 3. Upload ZIP archive to Fredhopper <a href="#fulldata-csv-stepthree-uploadziparchivetofredhopper" id="fulldata-csv-stepthree-uploadziparchivetofredhopper"></a>

Upload the zip file to the Fredhopper Managed Services environment that is given to you by your Technical Consultant using the 'fas' service interface.

Please note to include the checksum value in the request, as per the example below which is directed to the live1 instance.

{% code title="Example Request" overflow="wrap" %}

```
curl -D - -u "[cloud_username]":"[cloud_password]" -X PUT -H "Content-Type: application/zip" --data-binary @data.zip https://my.eu1.fredhopperservices.com/upload:live1/data/input/data.zip\?checksum=19afc8c99ec7ea27f0dba8e1c4fcff1f
```

{% endcode %}

Once the file has been uploaded, the system will send a simple HTTP response back with some important information contained within the header and body sections:

{% code title="Example Response" overflow="wrap" lineNumbers="true" %}

```
HTTP/1.1 100 Continue
 
HTTP/1.1 201 Created
Date: Fri, 21 Feb 2020 15:31:55 GMT
Server: Jetty(9.4.22.v20191022)
Location: https://my.eu1.fredhopperservices.com/upload:live1/data/input/2020-02-21_15-31-55
Content-Type: text/plain
Content-Length: 28
Connection: close
 
data-id=2020-02-21_15-31-55
```

{% endcode %}

{% hint style="warning" %}
You should capture the 'data-id' section in the HTTP response body that you receive from the API as this will be used in subsequent steps. The 'data-id' value is unique to every request.
{% endhint %}

### 4. Trigger for data to be loaded <a href="#fulldata-csv-stepfour-triggerfordatatobeloaded" id="fulldata-csv-stepfour-triggerfordatatobeloaded"></a>

Thus far, we have only uploaded the data to the Fredhopper Managed Services environment. Now, we shall initiate a trigger instructing Fredhopper to re-index with the new data and you must use the 'data-id' value that you captured in the previous step.

{% code title="Example Request" overflow="wrap" %}

```
curl -D - -u "[cloud_username]":"[cloud_password]" -X PUT -H "Content-Type: text/plain" --data-binary "data-id=2020-02-21_15-31-55" https://my.eu1.fredhopperservices.com/upload:live1/trigger/finalize
```

{% endcode %}

The HTTP response header that you receive back from our system at this stage contains a new 'Location' value, which we can use to monitor the status of the re-index:

{% code title="Example Response" overflow="wrap" lineNumbers="true" %}

```
HTTP/1.1 201 Created
Date: Fri, 21 Feb 2020 15:47:17 GMT
Server: Jetty(9.4.22.v20191022)
Location: https://my.eu1.fredhopperservices.com/upload:live1/trigger/finalize/2020-02-21_15-47-17
Content-Length: 0
Connection: close
```

{% endcode %}

### 5. Monitor the status of the re-index <a href="#fulldata-csv-stepfive-monitorthestatusofthere-index" id="fulldata-csv-stepfive-monitorthestatusofthere-index"></a>

The status value for the re-index job can be checked by sending the following command using the 'Location' value that you captured in the previous step:

{% code title="Example Request" overflow="wrap" %}

```
curl -s -D - -u "[cloud_username]":"[cloud_password]" https://my.eu1.fredhopperservices.com/upload:live1/trigger/finalize/2020-02-21_15-47-17/status
```

{% endcode %}

{% code title="Example Response" overflow="wrap" lineNumbers="true" %}

```
HTTP/1.1 200 OK
Date: Fri, 21 Feb 2020 15:49:09 GMT
Server: Jetty(9.4.22.v20191022)
Content-Type: text/plain
Content-Length: 57
Connection: close
 
RUNNING
Started execution at Fri Feb 21 15:47:17 UTC 2020
```

{% endcode %}

When complete you should get a success response.

{% code title="Example Response" overflow="wrap" lineNumbers="true" %}

```
HTTP/1.1 200 OK
Date: Fri, 21 Feb 2020 15:51:15 GMT
Server: Jetty(9.4.22.v20191022)
Content-Type: text/plain
Content-Length: 8
Connection: close
 
SUCCESS
```

{% endcode %}

Possible status codes returned are:

<table><thead><tr><th width="204">Status</th><th>Description</th></tr></thead><tbody><tr><td>Unknown</td><td>No known state yet: trigger has not yet been picked up</td></tr><tr><td>Scheduled</td><td>Trigger has been picked up, and will start execution soon</td></tr><tr><td>Running</td><td>Triggered job is running currently</td></tr><tr><td>Delayed</td><td>Triggered job is ready to run, but delayed (eg: due to insufficient capacity)</td></tr><tr><td>Success</td><td>Triggered job has finished successfully</td></tr><tr><td>Failure</td><td>Triggered job has failed</td></tr></tbody></table>

{% hint style="info" %}
When deciding on the frequency to poll the progress of the load-data processes from your-side, we recommend having a minimum interval of 60 secs in-between each check that you make.
{% endhint %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://crownpeak.gitbook.io/product-discovery/sending-and-managing-product-data/flatfiles/custom-datafeed/custom-data-uploads.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.