Limitations and Best Practices

The FHR A/B testing feature is a powerful tool, but there are a few guidelines and limitations to follow in order to be able to make the most out of it.

Polling Frequency/Caching Duration

It is best to cache the running A/B tests response around as long as you cache your regular FHR queries. If your usual expiration time is 20 minutes, you can cache the running A/B tests response just as long. If you want to have more up to date data or do no caching at all, you can stick to the default suggested minimum time of 5 minutes.

Trigger limitations

Unfortunately, not all triggers which are available for non A/B test entities, such as campaigns, can be used for A/B testing due to filtering limitations. Because of this, the list of available triggers for A/B tests will be slightly smaller. The available triggers are:

• Universe

• Locale

• User Location

• Navigation Steps

• Query Type

• Custom triggers:

  • Text - full support

  • Number - full support

  • Date - limited support: can be added, but will not be considered for filtering i.e. the trigger will not appear as part of the filters in the A/B test in the response when getting running A/B tests. However, this will still allow you to setup date/time triggers.

Hint: While the Keyword trigger is not supported, you can still add a search query as a user location, thus making use of search words as triggers.

A/B tests on the homepage

One other limitation is for setting up triggers on the homepage. When requesting the homepage in FHR, the fh_location field can be omitted, but this will cause issues. If you omit it, the response will be defaulted to a location for home with the default universe and locale set up. This makes it impossible to run filtering on the running A/B tests, so it is advisable that you always set the fh_location, including a minimum of //<universe>/<locale>, and check it against the running A/B tests when requesting the homepage.

Otherwise, the filtering wouldn't be able to tell if an A/B test can be run, therefore the A/B tests will not trigger for the homepage.

A/B tests on Product Detail Pages

Similarly to the homepage situation, you can omit the fh_location property for PDPs(Product Detail Pages). This again causes issues, as without it we're not able to test for universe and locale, which would make filtering impossible. So in order for filtering to work, you have to supply an fh_location including at least the universe and locale in order to make proper use of triggers on PDPs. This could be done by ensuring that the FHR query request for a PDP is set by convention to include an fh_location of //<universe>/<locale>.

Stick to the SDK and default implementations if possible

We've put effort in making the integration as seamless as possible through the SDK. The default implementations are meant to be the best way to go. While we've added the ability for custom implementations of certain aspects, we strongly advise that you adhere to the adhere to the default implementation as much as possible. We cannot guarantee that we can offer support for custom implementations of the SDK features such as the RunningAbTestsFetcher or AbTestsCache.