{"id":13593,"date":"2018-05-21T15:26:57","date_gmt":"2018-05-21T22:26:57","guid":{"rendered":"https:\/\/www.kochava.com\/?p=13593"},"modified":"2022-08-18T15:01:10","modified_gmt":"2022-08-18T22:01:10","slug":"a-b-n-and-multivariate-testing-in-push-messaging","status":"publish","type":"post","link":"https:\/\/www.kochava.com\/ko\/blog\/a-b-n-and-multivariate-testing-in-push-messaging\/","title":{"rendered":"The Role of A\/B\/n and Multivariate Testing in Push Messaging"},"content":{"rendered":"[vc_row type=&#8221;in_container&#8221; full_screen_row_position=&#8221;middle&#8221; column_margin=&#8221;default&#8221; column_direction=&#8221;default&#8221; column_direction_tablet=&#8221;default&#8221; column_direction_phone=&#8221;default&#8221; scene_position=&#8221;center&#8221; text_color=&#8221;dark&#8221; text_align=&#8221;left&#8221; row_border_radius=&#8221;none&#8221; row_border_radius_applies=&#8221;bg&#8221; overflow=&#8221;visible&#8221; overlay_strength=&#8221;0.3&#8243; gradient_direction=&#8221;left_to_right&#8221; shape_divider_position=&#8221;bottom&#8221; bg_image_animation=&#8221;none&#8221;][vc_column column_padding=&#8221;no-extra-padding&#8221; column_padding_tablet=&#8221;inherit&#8221; column_padding_phone=&#8221;inherit&#8221; column_padding_position=&#8221;all&#8221; column_element_spacing=&#8221;default&#8221; background_color_opacity=&#8221;1&#8243; background_hover_color_opacity=&#8221;1&#8243; column_shadow=&#8221;none&#8221; column_border_radius=&#8221;none&#8221; column_link_target=&#8221;_self&#8221; column_position=&#8221;default&#8221; gradient_direction=&#8221;left_to_right&#8221; overlay_strength=&#8221;0.3&#8243; width=&#8221;1\/1&#8243; tablet_width_inherit=&#8221;default&#8221; tablet_text_alignment=&#8221;default&#8221; phone_text_alignment=&#8221;default&#8221; animation_type=&#8221;default&#8221; bg_image_animation=&#8221;none&#8221; border_type=&#8221;simple&#8221; column_border_width=&#8221;none&#8221; column_border_style=&#8221;solid&#8221;][vc_column_text]Marketing is as much about retargeting as it is about user acquisition. As is commonly known, the cost to acquire a new user is much more than to retain an existing customer. Whether a marketer is re-engaging valuable users or retargeting ones at risk for <a href=\"https:\/\/www.kochava.com\/kochava-intelligence\/\">churn<\/a>, crafting multiple messages to see which one drives results best is key to successful marketing.\r\n\r\nIn this aspect of marketing, <a href=\"https:\/\/www.kochava.com\/engagement\/\">push messaging<\/a> is a valuable marketing tool. While it enables marketers to reach specific audiences with personalized content, poorly executed push campaigns can also negatively influence a brand\u2019s image and a user\u2019s perception of an app. As a remedy, marketers commonly perform A\/B\/n testing, or multivariate analysis, to determine which content is most compelling to their audiences.\r\n\r\nA\/B\/n testing (where n is the number of variables being tested) can be as simple or as in depth as preferred, but it plays a pivotal role in understanding an audience\u2019s perception of marketing message better. It also allows an audience to self-select what types of messaging they prefer to see.\r\n\r\nIn-house multivariate testing can be incredibly resource intensive. Marketers often target multiple audience segments, and ideally, they would test the messaging for each. If a marketer is targeting 10 different segments, each with 10 different variants, over 100 separate iterations must be weighted, balanced, and optimized against each other.\r\n\r\nA better approach is to use a push notification system that auto-optimizes campaigns in real time based on performance. This allows a marketer to reach an audience with as many variants as they choose per audience segment, without being concerned with the time investment of manually adjusting the weights based on performance.\r\n\r\nCreating campaigns driven by auto-optimization ensures that messaging is based on users\u2019 real-time responses.\r\n<h4>Create a data-driven push campaign<\/h4>\r\nMarketers can define an audience in Analytics or upload their own list of device IDs. <a href=\"https:\/\/www.kochava.com\/kochava-push-notifications\/\">Push messaging<\/a> is ideal for re-engaging users likely to <a href=\"https:\/\/www.kochava.com\/stop-losing-money-to-churn\/\">churn<\/a>. When defining an audience inside the Kochava platform, marketers can A\/B test manually or choose to automate their multivariate testing.\r\n<h4>A\/B test messaging<\/h4>\r\nTo simplify A\/B\/n testing, Kochava automatically optimizes toward the most successful message (variant), freeing the marketer to implement a large range of potential messaging variations without worrying about manually applying the prevailing message of the resulting data to their campaigns. The system has three machine learning algorithms to determine a winner. These include:\r\n\r\n<img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-13634\" src=\"\/wp-content\/uploads\/2018\/05\/ABC-Push-Continuous.jpg\" alt=\"Continuous A\/B\/n test\" width=\"1914\" height=\"700\" \/>\r\n<ul>\r\n \t<li>Continuous: constantly uses real-time feedback to adjust weighting based on the individual open rates of each variation<\/li>\r\n<\/ul>\r\n<img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-13636\" src=\"\/wp-content\/uploads\/2018\/05\/ABC-Push-Progressive.jpg\" alt=\"Progressive A\/B\/n test\" width=\"1914\" height=\"700\" \/>\r\n<ul>\r\n \t<li>Progressive: waits 48 hours, and adjusts weights based on the relative success rates of the variants; re-adjusts every 24 hours, until one variant has been weighted to 100%.<\/li>\r\n<\/ul>\r\n<img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-13635\" src=\"\/wp-content\/uploads\/2018\/05\/ABC-Push-Full-Redirect.jpg\" alt=\"Full redirect A\/B\/n test\" width=\"1914\" height=\"700\" \/>\r\n<ul>\r\n \t<li>Full Redirect: waits for 72 hours, then picks one as the \u201cwinner\u201d and gives it a 100% weighting<\/li>\r\n<\/ul>\r\nMarketers can still manually set or shift weights of ongoing campaigns to match their own testing methodology, if needed.\r\n<h4>What to test<\/h4>\r\nMarketers can test headlines and content messaging to observe what\u2019s working or not. Additionally, they should be selective in their variants by only testing one or two elements each to clearly distinguish what\u2019s most effective.\r\n\r\nA\/B test:\r\n<ul>\r\n \t<li>Headlines<\/li>\r\n \t<li>Push messaging content<\/li>\r\n \t<li><a href=\"https:\/\/www.kochava.com\/smartlinks\/\">Deep link destinations<\/a> (view in Analytics)<\/li>\r\n \t<li>Offers<\/li>\r\n<\/ul>\r\nOften, in multivariate testing, there are several individual variants being tested, such as three different message types and three different headlines. The system then tests the variants together to see which combination works best.\r\n\r\nEngagement automatically tests based on one of the three configurations selected and operates under the best practices of:\r\n<ul>\r\n \t<li>Testing on the same day<\/li>\r\n \t<li>Choosing winner on based 100% for best statistical significance<\/li>\r\n<\/ul>\r\nObserve what content gets their attention. Is it a more personal message crafted based on their demographic? Is it a certain type of offer? Test messaging, see what prevails, modify, and then test again.\r\n<h4>Measure, measure, measure<\/h4>\r\nMarketers can follow audiences down the funnel to see how they perform, and get further evidence on the success of the messaging that won in the multivariate testing. Every campaign is another attempt to refine content based on data, test what works, and see results in the form of conversions.\r\n\r\nWant to know more? Visit our <a href=\"https:\/\/support.kochava.com\/engagement-overview\/engagement-campaign-overview\" target=\"_blank\" rel=\"noopener noreferrer\">support documentation<\/a> about using A\/B\/n automation.[\/vc_column_text][\/vc_column][\/vc_row]","protected":false},"excerpt":{"rendered":"<p>[vc_row type=&#8221;in_container&#8221; full_screen_row_position=&#8221;middle&#8221; column_margin=&#8221;default&#8221; column_direction=&#8221;default&#8221; column_direction_tablet=&#8221;default&#8221; column_direction_phone=&#8221;default&#8221; scene_position=&#8221;center&#8221; text_color=&#8221;dark&#8221; text_align=&#8221;left&#8221;&#8230;<\/p>\n","protected":false},"author":55,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[11,27],"tags":[],"class_list":{"0":"post-13593","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-news-and-updates","7":"category-education"},"_links":{"self":[{"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/posts\/13593","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/users\/55"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/comments?post=13593"}],"version-history":[{"count":7,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/posts\/13593\/revisions"}],"predecessor-version":[{"id":45202,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/posts\/13593\/revisions\/45202"}],"wp:attachment":[{"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/media?parent=13593"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/categories?post=13593"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kochava.com\/ko\/wp-json\/wp\/v2\/tags?post=13593"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}