Horse racing and betting go hand-in-hand, so when TwinSpires.com, official Advance Deposit Wagering (ADW) platform for the Kentucky Derby, was looking for an avenue to boost bets from fans, mobile was an obvious choice. They launched the TwinSpires app a few years ago to a healthy following. This year’s install rates seemed exceptionally impressive—until they received an abnormally high invoice that led them to dive deeper into the data behind those installs.
Since 1875, the Kentucky Derby has been held at Churchill Downs; this 143-year pedigree was a huge advantage when they launched the TwinSpires app wagering platform for connected devices to complement their web-based product.
“Based on our initial success last year, this year’s results seemed to build on that. But our costs were also escalating, meaning we quickly realized that something was wrong,” said a TwinSpires spokesman.
The conversion rate that was notTwinSpires expected mobile installs to peak at the height of the Kentucky Derby and experienced a marked increased from their first year. At first glance, this too was expected since the TwinSpires name is well established with the race and it was not a new launch. What wasn’t expected was the exorbitant click volume and inflated click-to-install rate (CTI).
The cause behind the high cost of user acquisition was an inflated CTI, which for 2017 was several times greater than the previous year. In addition, many of those installs did not convert, meaning that the activity stopped there with no additional post-install events (racetrack bets) completed.
With a lower conversion rate versus the previous year’s, there were questions as to whether some of those installs were organic. Organic sniping is suspected when there is an extremely high CTI ratio and when a network’s (or sub-publisher’s) install wins closely mirror in-app conversion performance (i.e., quality of installs) of organic installs.
Fraud Console reports show true campaign activityThe Kochava Fraud Console exposed all the transactions that occurred during the campaign—both the good and the bad. A post-campaign analysis of TwinSpires activity using the Kochava anti-fraud algorithms identified the data outliers highly indicative of malpractice. Kochava detected IP addresses and device IDs with high click volume, click/install geographic outliers, extremely short mean-time-to-install rates, and ad stacking.
Diving deeper into the data showed that things were not as they seemed. By breaking down CTI by network, Kochava showed discrepancies in the number of clicks recorded and what TwinSpires.com’s partner networks reported.
A number of networks were also involved in click delta disparities: a proportion of installs occurred in states where online gambling is prohibited. On top of that, the associated installs showed little action beyond the install.
“This was an informative exercise demonstrating how certain entities are able to manipulate programmatic advertising for their benefit and not that of their client,” the TwinSpires spokesman said.
Why clicks matter, why attribution mattersIn the analysis of TwinSpires.com traffic, the Kochava data science team showed the networks carrying the highest click rates and lowest install numbers. Without recording clicks, those install numbers would have looked legitimate.
Reviewing the CTI rate more closely showed that those networks weren’t truly driving installs—they were gaming the system. In other words, several networks were guilty of click sniping (flooding their channel with clicks until a user installed) in addition to malware that accounted for the bogus installs. TwinSpires was able to use the Fraud Console reports to receive make-goods from several of their partners. Moving forward, knowing which partners drive quality installs, where users complete post-install events, is more important than having partners tout a high number of clicks as proof that their work was successful.
“The Fraud Console revealed different values of our network partners and that we can’t always take our conversion rate at face value,” their spokesman said.
Solutions: Vetting media partners and preventing fraudIn addition to employing the Fraud Console, the Kochava data science team recommended using Traffic Verifier, Alerting, Configurable Attribution, and What if? tools by Kochava for planning future app campaigns.
With Traffic Verifier, a frequency cap on the number of clicks deemed acceptable can be set. Anything beyond the set threshold would not be attributed. With the Alerting tool, they can set up notifications for when a volume cap is reached. Using Configurable Attribution, they can create shorter lookback windows to segment their higher value users—those who installed and used the app early in a campaign. Finally, the What if? tool allows an advertiser to run campaigns in an experimental mode before launching the actual campaign. This way, they can see the outcomes of setting the frequency caps or running with different networks.
With so many players involved in media buys and ad placement, fraud is inevitable and ever-evolving. The Kochava Fraud Console detects and mitigates the behavior of suspicious entities acting outside of normal behavior, such as what TwinSpires experienced. Using these tools, they can move forward in running campaigns knowing that they can access and provide evidence of any malpractice to hold their partners accountable.