Session-Based Multivariate Testing
Unleashing the Power of Session-Based Testing for Advanced Reporting in Google Ad Manager
In the ever-evolving world of ad tech, the ability to conduct precise and insightful testing is key to staying ahead. Traditional methods often fall short when it comes to the granular data required for comprehensive optimization. Enter the Session-Based Testing Pipeline—a powerful method designed to bring advanced reporting capabilities to Google Ad Manager (GAM) through the use of session-level key values. This approach enables detailed multivariate testing, offering insights far beyond what native GAM metrics like URL or page load speed can provide.
Key Concepts and Definitions
Session-Level Key Values:
Session-level key values are unique identifiers that are set and tracked at the session level using cookies. These values are crucial for monitoring and correlating data across multiple test variations during a user’s session. Once set, these key values are sent to various platforms, including Google Analytics (GA), Google Ad Manager (GAM), event collection servers, and others like Segment and Assertive Yield. This enables comprehensive tracking and analysis of user behavior and ad performance.
Variants:
In multivariate testing, a variant refers to one of the multiple versions of an element (such as an ad unit or configuration) being tested. Each variant is tracked using session-level key values, allowing for a detailed comparison of their performance across different user sessions.
Buckets:
Buckets are groups of session-level key values assigned to different variants in the test. By using multiple buckets for each variant, you can assess the variability (or “noise”) within a variant group, which helps in determining whether the differences in performance between variants are statistically significant.
Pipeline:
The pipeline refers to the entire process of assigning session-level key values, collecting event data, pulling data from GAM, and merging it all into a cohesive dataset. This pipeline ensures that the data is consistently processed and analyzed, providing accurate insights into the performance of each variant.
Multivariate Testing:
Multivariate testing allows for the simultaneous execution of multiple tests, each with its own set of variants. The assigned session-level key values make it possible to track how different test variations perform within the same user session, offering a comprehensive view of the impact each variant has on metrics like impressions and revenue.
Data Merging:
The pipeline’s ability to merge event data with GAM reports ensures that the data collected is not only detailed but also highly accurate. This merging process allows you to produce metrics that reflect the true performance of each test variation, leading to more informed decision-making.
How the Method Works
Assigning Session-Level Key Values:
The process begins by setting session-level key values using cookies. While 5000 is a common example, the actual number of key values can vary depending on the complexity of the test. These values are then mapped to a table of multivariate tests, allowing for precise tracking of each user session. Once set, the same key values are sent to various platforms, including GAM, GA, event collection servers, Segment, and Assertive Yield. This ensures consistency across data sources and provides a broader view of your ad performance.
Event Data Collection:
During each user session, various data points—including the session key value—are collected and sent to the event collection server. This aggregated data is essential for measuring test performance across different segments. However, not all users will have access to event-level data collection. For those relying solely on GA and GAM, the session-level key values can still be used effectively to track performance metrics and analyze the impact of different test variants.
Daily Data Pull from GAM:
Data is pulled daily from GAM using its API, with dimensions set for the day and session key value. This data includes crucial metrics like revenue and impression counts for each test bucket, providing a snapshot of how each test variant is performing.
Multivariate Testing Framework:
The session key values enable the execution of complex multivariate tests. For example, you can set up multiple 10-way tests or a single 5-way test depending on your needs. Using multiple key values for each variant allows you to measure the noise within each bucket, helping you determine whether the differences between variants are significant enough to validate the test’s effectiveness.
Data Merging:
Once the data from GAM is pulled, it is merged with the event collector data based on the session key value. For those relying on GA and GAM, the data merging process can still provide high-fidelity insights, even if it doesn’t offer impression-level granularity. By aggregating data at the bucket level, you can still gain a comprehensive understanding of how each variant performs.
Handling Discrepancies and Scalability
As with any complex system, discrepancies can arise. The Session-Based Testing Pipeline is designed to handle known discrepancies, such as those caused by GAM filtering invalid traffic (IVT) or event collection server failures. Additionally, impressions occurring near the day divider may be attributed to different days between systems. To manage these, a separate diagnostics report is generated to identify and investigate any discrepancies.
Scalability is another key feature of this method. The event collector and data pipeline merging processes are designed to scale effectively, ensuring that even with high data volumes, the system remains robust and reliable. The merged data is stored in a database that is easily accessible to data scientists, stakeholders, and technicians for further analysis and integration with other applications.
Conclusion
The Session-Based Testing Pipeline offers a powerful way to gain deeper insights into your ad performance through advanced multivariate testing. By focusing on session-level key values, this method provides high-fidelity data that can drive smarter, more effective optimizations. The added capability to send these values to platforms like Google Analytics, Segment, Assertive Yield, and more expands the flexibility and scope of your analytics, making it easier to integrate and utilize across various tools in your stack. While it may not offer insight into granular data unless specifically set up beforehand, the overall benefits in terms of precision and scalability make it an invaluable tool for any ad tech strategy.
Whether you have access to event-level data or rely solely on GA and GAM, the Session-Based Testing Pipeline can elevate your ad tech capabilities, uncover hidden opportunities for optimization, and ensure that your testing efforts yield actionable results.
If you need help designing or implementing this data pipeline, or if you want to discuss other data pipeline solutions, reach out. Let’s work together to unlock the next level of success in your ad tech strategy.