Google Analytics receiving transaction data more than once - Paypal issue?

We turned on the standard Paypal integration in January and since then have had a real issue with transaction data being duplicated in Google Analytics.

Where we used to have maybe 3 to 6 duplicated transactions per month, we had 29 transactions in Jan duplicated by Analytics - presumably because our website is resubmitting the data. By this I mean a transaction occurs once but google analytics records the transaction when it originally occurs, but then it repeats it again - maybe later the same day or a different day. We had a transaction on 7th Feb that reappeared twice on 8th Feb, 3 times on 9th Feb and so far once today. Thats 7 times, and just an example! This one is massively skewing our turnover and product statistics as it happened to be a large order - and surely the customer is not still pressing buttons when they shouldn’t 3 days later…

Anyone know why or how this happens? I’ve looked at a few of the duplicates and they are different browsers that the customers are using. The common thing mostly is they paid by Paypal.

Any help would be appreciated.

Thanks,
Emma

Any insight on this, please, @franclin?

Hi @emma

Apologies for the misleading results that you are getting for those Paypal orders.

Since, we handled Paypal standard slightly differently than other payment gateways, situations like these seem to popup from time to time.

We are working on a fix for this issue right now and you can expect it to be rolled out by the end of this week.

Thanks once more for bringing this into our attention.

Kindest regards,
Franclin

Hi @emma

Please be advised that a fix for this issue has now been released and should be available straight away.

Please let us know if you are still encountering this problem.

Thanks a million for your custom and patience.

Kindest regards,
Franclin

Hi Franclin,

We are continuing to see this issue - albeit there are possibly fewer now. Today the original transaction I mentioned in this post (which is #1454875214) has landed in Analytics. This transaction happened on 7th Feb and has now posted itself into analytics 12 times - twice of which has been since you have issued a fix. What can we do to stop this; and can you reverse it out?

Emma

Hi Emma,

There is no way to send data deletion requests to Analytics, AFAIK.

@franclin perhaps we can discuss in our meeting today? Curious as to know WHEN it calls back to Analytics. Maybe we can add an Analytics flag to the WebOrders table, so we can’t duplicate a call?

Thanks,
Donogh

Hi @emma,

Sorry to hear that you are still experiencing this issue. We rely on a mechanism known as the IPN (Instant Payment Notifications) which are essentially asynchronous HTTP requests that Paypal posts to remote servers to inform them of a change of an order status. The most plausible explanation for the duplication of orders in your Analytics is that for some reasons, Paypal is sending several IPN requests to us which is a little odd.

In order to help us investigate this issue better, could you please tell us the number of retries that was attempted for this transaction? To locate those, please go to your Paypal account under History → IPN History. Please locate the IPN corresponding to that transaction (its paypal ID is 608486604H6886203). I have included some screenshots to help you further. I will be grateful if you could tell us what was the HTTP response code that Paypal reported, also the number of retries and the delivery status. Please kindly note that Paypal only reports transactions that are not older than 28 days.

Finally, I would like to confirm Donogh initial assessment in that we cannot submit requests to Google to remove duplicates. The reason for this is that the actual request is sent by the Google libraries and for obvious reasons, the implementation is hidden away from the clients. We can, of course, reverse engineer the process but we are not able to create new commands on the fly as they will certainly be rejected. So we are back to the ante.

Moving forward, once you have come back to us with the information that we asked you, we will start to investigate a much more sustainable approach to deal with this problem. As it is intermittent by nature, it may take time to nail it down so your patience will be greatly appreciated meanwhile.

Have a great weekend.

Kindest regards,
Franclin


Hi @Franclin,

Here is the information you requested from paypal, nothing looks untoward as far as I can tell.

Message ID: 8HL74897J6082973H
Date/time created: 07/02/2016 20:02 GMT
Original/Resent: Original
Latest delivery attempt date/time: 07/02/2016 20:02 GMT
Notification URL: https://secure2.nitrosell.com/www.broughtons.com/store/go/checkout-callback/8/
HTTP response code: 200
Delivery status: Sent
No. of retries: 0
Transaction ID: 608486604H6886203
IPN type: Transaction made

The transaction has once again arrived in Analytics since my last post (its now duplicated 13 times). Just for the record, we don’t get a duplicated order confirmation email each time this occurs.

Emma

Thanks for sharing that information with us.

We will keep working on a fix and as soon as possible there is any development we will let you know.

Kindest regards,
Franclin

Hi,

I understand it may take some time to fix properly, but is there anything you can do specifically about transaction #1454875214 from Feb 7th. Even today its landed twice in google - when will it stop!! I was really hoping not to have another month of messed up stats! If that one transaction stopped re-reporting, it would be a massive help.

Emma

Hi @emma

We just enabled some logs to catch up that behaviour.

If you could report another transaction in the meantime, that would be very useful.

Out of curiosity, how many duplicated have you noticed recently?

Regards,
Franclin

Hi @franclin_foping,

The following are some other transactions that have presented themselves to Google Analytics more than once in March:

1457049433 (twice)
1457201194 (twice)

Other transactions that have duplicated more than twice since Feb and may therefore pop up again:

1454080732 (8 times)
1455036124 (5 times)
1456164188 (3 times)

They are not always easy to spot unless you go looking for them as they are often not on the same day as the original, but the incidence has definitely reduced.

Emma

Hi @emma

Thanks for providing that insight. We are currently working on a fix for that issue.

So please watch out this space for an update.

kindest regards,
Franclin

Hi @franclin,

To help with your diagnostics, I wanted to let you know that the original persistent transaction from early Feb - 1454875214 has landed again today 10/3/16 - now a grand total of 17 times!

Emma

Hi @emma

Thanks for providing further insight. Apologies for the inconvenience.

Hopefully the fix should be in place to prevent this issue in time for the weekend.

Regards,
Franclin

Hi @emma

Please be advised that we released a fix for this issue last Friday. We also added some logs to track this sort of behaviour in the future. Should you come across any duplicated order next time, please let us know so that we can look into it for you.

Apologies for the inconvenience once more.

Kindest regards,
Franclin

Hi @franclin,

We have some more duplicated transactions, these are the ones since Monday (14th March):
1458216036
1458156173
1458176226
1458137237

Hi @emma

Thanks a lot for your feedback. Our server side logs are showing that for all the 4 orders that you reported, we rendered the pixel only once. It therefore seems like the duplicate requests are sent from the front-end.

The obvious explanation is that customers are most likely bookmarking or saving the order confirmation page and whenever they reopen the page the Google pixel is still present for some reasons.

We are definitely going to address that drawback as we have now got to the bottom of the problem.
Please watch out this space for an update.

Thanks a million for your patience.
Regards,
Franclin

Hi @franclin,

We’d definitely appreciate a timely fix for this. It does seem worse since your last patch.

There have been 34 incidences in March alone so far. This one has repeated 7 times: 1458176226 - 3 of which were yesterday.

Analytics will once again be thousands of pounds over reporting this month which causes such a headache for us.

Emma

Hi @emma

So far, our patches exclusively focused on the back-end side but now that we know that it is happening at the front-end, we will shift our attention there. Thus, there is a good chance that we will nail down this issue very soon.

I can confirm that according to our logs, the GA pixel for order 1458216036 was sent only once (on March 17 at exactly 01:02:49). Can you please tell us at which time the other 3 calls were made yesterday?

Thanks,
Franclin