Transaction Deduplication Function

Marketing tools capture more data than ever and thus must have good routines to separate the good from the bad signals. As a tag management expert it is your job to help and provide those parameters required to make good decisions. Often it will mean to send a conversion signal to a network only once it occurred. Which is a very common scheme in tag management daily business. With JENTIS you now have a more resilient tool to handle this job.

In this guide we will show you how to implement an efficient way to handle deduplication with JENTIS server side. Saving resources frontend and have a stable process that runs smoothly server side.

Filtering Data

Our goal at hand is to check if a conversion occurred first, before actually submitting the signal and parameters to a network. For this a unique identifier of that conversion is required, which is on transaction based conversions the according transaction_id. So for this guide we will use a common example of an ecommerce store. Where the conversion is a purchase and has a unique id with each submitted to our data layer.

Before we dig deeper into the “how” we should also shortly discuss the “why”. Some marketing tools are not capable of deduplication in their internal processes, so the tracking has to filter out duplicates. That happens often on page reloads (such as closing the website app on your smartphone after purchasing and later re-opening the same app – which will reload this recent page and unintentionally send a redundant conversion signal). To filter that out before the conversion is send we can implement a short routine.

Server Side Deduplication with JENTIS

With all the basics covered lets jump head first into the implementation. Here is a short summary of what is required and what the parts in the tag management are.

We will create those elements:

• Trigger: To actually only capture the first conversion and no more the triggers will need a parameter to check this back with.
• Backend Variable: This will be the parameter for the trigger. The actual result value must be either true or false, signaling if this is a new or redundant conversion.
• Frontend Unique Identifier: An ID, such as a transaction_id, that identifies the conversion.
• Tags: Update the tags to use the new trigger and maybe create new tags that connect to the logic of deduplication.

Here are the details and step by step integration of those. We will start with the variables and later update the trigger and tags.

Frontend Unique Identifier

The unique identifier is core of this process. So an external application is required that knows what a conversion was related to (ie. a transaction) and submits that to the data layer. This is a common standard with ecommrece tracking, such as Universal Analytics enhanced ecommerce.

Using the JTM native data model this data may be pushed to the JENTIS data layer in this fashion:

_jts.push({
  "track"  :  "order",
  "orderid":  "abcdef1234567890",
  ...
});

With that we can access the value from the standard variable “order_id” (which is a JTM default frontend variable). This we can access on the server side with backend variables.

Backend Variable

The main chunk of code comes with this backend variable. So we will look at it line by line and see what it does.

async function() {
  var storage_name  = "existing_order_ids";
  var max_storage   = 5;
  let existing_ids  = await this.storage.read(storage_name) || "";
  let new_order_id  = this.getFrontendVariable("dl_order_number");

  if(existing_ids && new_order_id != null && existing_ids.indexOf(new_order_id) >= 0){
    return true;
  } else if(new_order_id && new_order_id != null) {
    existing_ids += new_order_id+";";

    if(existingIds.split(";").length > max_storage){
      existing_ids = existing_ids.split(";");
      existing_ids.shift();
      existing_ids = existing_ids.join(";");
    }

    this.storage.write(storage_name, existing_ids, Date.now()+94608000000);

    return false;
  }
  return null;
}  

To meet our goal we will store a list of known conversion or transaction ids. This is saved on the servers local storage. The storage_name is the location, just a unique static key to identify the location (that is not used in a different variable or function). We will also define a maximum list length, so the list length is limited and not poluting our storage. To access previously saved ids the code uses the API this.storage.read(storage_name). For matching a new id to the existing list we need to read a new value with this.getFrontendVariable(…). Here you should use the reference from the previous step (creating the frontend variable that holds the conversion identifier).

Now the rest is to simply if-else the situation. The variable will return a “true” as a signal to the trigger, telling that this conversion at hand is new, if the new id is found in the existing ids list. Otherwise the return value will be “false” and this new id will be concatenated to the existing list and the storage will be updated (using this.storage.write oparation).

That is all and we can save this function as a new backend variable.

Trigger Update

Now we need to actually apply the logic in a trigger. We will add a condition on the “Order” conversion trigger for that.

Tag Update

The last step is to actually activate a tag on our new trigger. Two scenarios are plausible here: track the conversion only the first time and optionally add a tag to track redundant calls as an event (that is not a conversion).

So if you need to have a safety or just curious to have all the data (meaning to also track events when redundant conversions would have been tracked but were filtered by our setup here) an accompanying tag makes sense. Just to make sure you are not missing out on anything and the actual process of deduplication is activated correctly. Thus you’d need a second trigger that has the same conditions, but the check on “Order ID Existing” is looking for “false”.

Conclusion

Make sure to preview this new process in the JTM preview. If everything looks good to go – you are ready for the release!