\r\n\r\nFor a long time, creative has been the holy grail of advertising. Something that is a pure art and cannot be quantified or optimized. Even with the proliferation of digital media and cheap computational power, optimization was something that was done after the ad creative was finalized. But this basic premise is being increasingly challenged.\r\n\r\nAt Aarki, our data scientists are constantly trying to find innovative ways to improve ad performance. Over the past five years, they have developed several industry leading technologies to deliver greater value to mobile app marketers. These technologies have been used on 120,000+ campaigns to serve almost 14 billion impressions.\r\n\r\nIn this article we share the results of a robust statistical evaluation of the performance of one such technology — multivariate creative optimization.\r\n\r\nSample Data<\/strong>\r\n\r\nThe data for this analysis is based on a random sample consisting of 381 distinct campaigns that were run by Aarki during the past six months. To protect the identities of our clients, we don\u2019t share any campaign-specific data in this article. These campaigns represent a diverse mix consisting of both game and nongame mobile apps, campaigns of different budget sizes, and across a wide\u00a0geographical area. In this sense, they are representative of Aarki\u2019s client base.\r\n
Analysis Methodology<\/h3>\r\nData was initially manipulated in PostgreSQL and then analyzed in R. The primary objective of the analysis was to evaluate the incremental performance improvement delivered by multivariate optimization of the ad creative. For the purpose of the analysis, the first creative combination was assumed to be the baseline and performance of subsequent creative combinations was analyzed in relation to the baseline combination. Performance of each combination was quantified by dividing the total media cost by\u00a0the number of users acquired, i.e.,<\/em> cost per install (CPI).\r\nResults: Overall CPI Improvement<\/h3>\r\nThe analysis indicates an average CPI improvement of 41.2% as a result of multivariate creative optimization compared to the baseline creative. Results of a Student\u2019s t-test (shown in Table 1) validate this finding to a 99.99%+ confidence with a t-statistic of 6.28. Given that in every case the baseline creative was developed using current ad design best practices, this is a very compelling finding.\r\n\r\nTable 1: Results of a Student\u2019s t-test on CPI Improvement<\/strong>\r\n\r\n![\"blog_table1\"](\"https:\/\/assets.kochava.com\/kochavacom\/uploads\/2016\/07\/blog_table1.png\")
<\/a>\r\n\r\n \r\nResults: Key Drivers of CPI Improvement<\/h3>\r\nTo further understand the main drivers of CPI improvement, we conducted a stepwise generalized least squares analysis on the dependent variable. Results of this analysis are shown in Table 2.\r\n\r\nTable 2: Generalized Least Squares Model of the Drivers of CPI Improvement<\/strong>\r\n\r\n![\"blog_table2\"](\"https:\/\/assets.kochava.com\/kochavacom\/uploads\/2016\/07\/blog_table2.png\")
<\/a>\r\n\r\n \r\n\r\nThe model has an overall adjusted R-squared value of 0.604, which represents a fairly decent goodness of fit. It suggests that the four drivers identified in the model are able explain the changes in CPI to a high degree, even within the assumptions\u00a0of a parametric linear construct and error normality. Since the objective of this analysis was to understand the tendencies and significance of the drivers, and not to build a predictive model, we did not attemp at\u00a0alternate modeling formulations.\r\n\r\nExamining the t-statistics of the four explanatory variables indicates that all of them are significant at the 85% confidence interval.\r\n\r\nThe biggest drivers of CPI improvement are imp_vol<\/em> (total volume of impressions served) and cpi_base<\/em> (CPI of the baseline creative). This suggests that Aarki\u2019s multivariate creative optimization works and is able to deliver progressively higher improvements in performance as more and more impressions are served. And the higher the baseline CPI, the greater the potential for performance improvement. The number of creative combinations tested is important but not as critical in driving improvements as the other two variables.\r\n\r\nVariability in the CPI is also an important factor. Our analysis indicates that the more the CPI varies across a campaign, the lower the likelihood of achieving greater performance improvements. This is an important insight. It proves the stability of our optimization algorithm and suggests that performance variability is likely due to exogenous factors outside the control of the campaign team. This implies that a consistent media strategy often delivers better results than one that varies a lot.\r\nSummary<\/h3>\r\nIn this article, we have presented the results of a robust statistical analysis of the role of Aarki\u2019s multivariate creative optimization algorithms in driving campaign performance. The analysis\u00a0indicates that Aarki’s technology delivers significant performance improvements, provided the campaign has a\u00a0robust impression volume and there is consistency in the media strategy.\r\n\r\n
\r\n\r\nAarki is a Kochava-certified network. Click here to learn how to create an install campaign<\/a>\u00a0or\u00a0reengagement campaign<\/a>\u00a0within Kochava.[\/vc_column_text][\/vc_column][\/vc_row]","protected":false},"excerpt":{"rendered":"[vc_row type=”in_container” full_screen_row_position=”middle” scene_position=”center” text_color=”dark” text_align=”left” overlay_strength=”0.3″ shape_divider_position=”bottom”][vc_column column_padding=”no-extra-padding” column_padding_position=”all”…<\/p>\n","protected":false},"author":1,"featured_media":9244,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[27,11,201],"tags":[],"class_list":{"0":"post-4238","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-education","8":"category-news-and-updates","9":"category-product-updates"},"_links":{"self":[{"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/posts\/4238","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/comments?post=4238"}],"version-history":[{"count":6,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/posts\/4238\/revisions"}],"predecessor-version":[{"id":33791,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/posts\/4238\/revisions\/33791"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/media\/9244"}],"wp:attachment":[{"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/media?parent=4238"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/categories?post=4238"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/tags?post=4238"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
<\/a>\r\n\r\n \r\nResults: Key Drivers of CPI Improvement<\/h3>\r\nTo further understand the main drivers of CPI improvement, we conducted a stepwise generalized least squares analysis on the dependent variable. Results of this analysis are shown in Table 2.\r\n\r\nTable 2: Generalized Least Squares Model of the Drivers of CPI Improvement<\/strong>\r\n\r\n<\/a>\r\n\r\n \r\n\r\nThe model has an overall adjusted R-squared value of 0.604, which represents a fairly decent goodness of fit. It suggests that the four drivers identified in the model are able explain the changes in CPI to a high degree, even within the assumptions\u00a0of a parametric linear construct and error normality. Since the objective of this analysis was to understand the tendencies and significance of the drivers, and not to build a predictive model, we did not attemp at\u00a0alternate modeling formulations.\r\n\r\nExamining the t-statistics of the four explanatory variables indicates that all of them are significant at the 85% confidence interval.\r\n\r\nThe biggest drivers of CPI improvement are imp_vol<\/em> (total volume of impressions served) and cpi_base<\/em> (CPI of the baseline creative). This suggests that Aarki\u2019s multivariate creative optimization works and is able to deliver progressively higher improvements in performance as more and more impressions are served. And the higher the baseline CPI, the greater the potential for performance improvement. The number of creative combinations tested is important but not as critical in driving improvements as the other two variables.\r\n\r\nVariability in the CPI is also an important factor. Our analysis indicates that the more the CPI varies across a campaign, the lower the likelihood of achieving greater performance improvements. This is an important insight. It proves the stability of our optimization algorithm and suggests that performance variability is likely due to exogenous factors outside the control of the campaign team. This implies that a consistent media strategy often delivers better results than one that varies a lot.\r\nSummary<\/h3>\r\nIn this article, we have presented the results of a robust statistical analysis of the role of Aarki\u2019s multivariate creative optimization algorithms in driving campaign performance. The analysis\u00a0indicates that Aarki’s technology delivers significant performance improvements, provided the campaign has a\u00a0robust impression volume and there is consistency in the media strategy.\r\n\r\n
\r\n\r\nAarki is a Kochava-certified network. Click here to learn how to create an install campaign<\/a>\u00a0or\u00a0reengagement campaign<\/a>\u00a0within Kochava.[\/vc_column_text][\/vc_column][\/vc_row]","protected":false},"excerpt":{"rendered":"[vc_row type=”in_container” full_screen_row_position=”middle” scene_position=”center” text_color=”dark” text_align=”left” overlay_strength=”0.3″ shape_divider_position=”bottom”][vc_column column_padding=”no-extra-padding” column_padding_position=”all”…<\/p>\n","protected":false},"author":1,"featured_media":9244,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[27,11,201],"tags":[],"class_list":{"0":"post-4238","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-education","8":"category-news-and-updates","9":"category-product-updates"},"_links":{"self":[{"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/posts\/4238","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/comments?post=4238"}],"version-history":[{"count":6,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/posts\/4238\/revisions"}],"predecessor-version":[{"id":33791,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/posts\/4238\/revisions\/33791"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/media\/9244"}],"wp:attachment":[{"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/media?parent=4238"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/categories?post=4238"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/tags?post=4238"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\r\n\r\nAarki is a Kochava-certified network. Click here to learn how to create an install campaign<\/a>\u00a0or\u00a0reengagement campaign<\/a>\u00a0within Kochava.[\/vc_column_text][\/vc_column][\/vc_row]","protected":false},"excerpt":{"rendered":"
[vc_row type=”in_container” full_screen_row_position=”middle” scene_position=”center” text_color=”dark” text_align=”left” overlay_strength=”0.3″ shape_divider_position=”bottom”][vc_column column_padding=”no-extra-padding” column_padding_position=”all”…<\/p>\n","protected":false},"author":1,"featured_media":9244,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[27,11,201],"tags":[],"class_list":{"0":"post-4238","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-education","8":"category-news-and-updates","9":"category-product-updates"},"_links":{"self":[{"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/posts\/4238","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/comments?post=4238"}],"version-history":[{"count":6,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/posts\/4238\/revisions"}],"predecessor-version":[{"id":33791,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/posts\/4238\/revisions\/33791"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/media\/9244"}],"wp:attachment":[{"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/media?parent=4238"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/categories?post=4238"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/tags?post=4238"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}