Requires Analyst plan or higher

Walkthrough Video

Need for New Report

Report Methodology

Report Parameters

Report Explanation

Estimate Toggle

How to Use

Our existing survey reporting utilizes the submission timestamp for the individual question response. We structured that report intentionally to confine survey results to the date range in which they were submitted. Doing it based on other timestamps created confusing views of the data that resulted in more confusion than clarity.

However, for some use cases, the response submission timestamp was not helpful. For instance, when measuring daily attribution, a customer can submit their survey response the day after they placed their order, resulting in their response being recorded in the daily bucket that differed from the day their order was placed. This leads to a lot of variability when using survey responses to track daily attribution. So we wanted to do something about it!

This report takes the same data and packages it in a slightly different view that correlates survey response data to order data. We believe this will be more useful for order attribution estimation and analyzing data for marketing spend decisions.

Here's how we have built the report (find it in the newly created Advanced Reports tab in our app).

We pull orders based on their timestamp (must be placed during the date range) and survey view (survey must have loaded when they viewed the confirmation page) for the survey selected in the report parameters. Then we automatically associate the order total with each order.

Note: Surveys not associated with an order (i.e. survey via link) or orders without a survey view (i.e. subscription orders after the initial subscription), will not be included in the order list.

We connect the response to the survey question set in the report parameters with the associated order. The response to the survey does not need to happen during the date range from the report parameters, it only needs to exist.

Then in the Responses section of the report, we list out all responses, the % of total responses submitted, the order count for total responses submitted, the revenue associated with each response, and the AOV for the response. This is hard data that corresponds directly to the connections between the response submitted, order count, and revenue total.

The report parameters are what you set to view the data on the report according to your preferences.

Survey: The survey you are running on your store

Question: The question on your survey for which you want to see order attribution data

Audience: The audience you want to use to filter the data on the report. We automatically set this to the audience (if any) that is being used to target the survey you select.

Date Range: The date range during which the order needs to be placed on your store

Audience Notes: Customers can change audiences after responding to a survey. This happens often with new vs. returning customers. For instance, the following survey is targeted to New Customers, and in total was shown to 2,725 orders during March.

But when the audience filter for New Customers is applied, the Total Orders count decreases slightly.

Customers have made another purchase after their first order and have transitioned from the New Customer audience to the Returning Customer audience. Even though they were a new customer when they placed the order and completed the survey, they now are part of a different customer segment.

Just be mindful of this whenever you're running the report and making comparisons to the order data.

Total Orders: The total number of orders placed and matched to the survey set in the parameters

Total Revenue: The total revenue of the orders matched with this survey.

Total Orders with Response: The number of orders matched with this survey with a response to the question set in the parameters

Attribution Rate: Total number of orders with a response divided by the total orders

The default view of the report is the hard data; no modeling or estimation happens anywhere at the beginning. The screenshot below shows one such report with the hard data.

*However* unleash the power of the report by toggling the Estimate 100% toggle on!

The report divides order count and revenue totals by the attribution rate for all Responses and each survey question response. The % of Total and the AOV won't change. However, the Orders & Revenue columns will essentially create an estimate for total orders and total revenue for the responses as if 100% of your customers who were shown your survey had responded to your survey at the same proportions.

Many brands already use something like this with their survey response rate to track and identify marketing channel trends. However, the survey response rate combined with the attribution data in a daily report will not be as accurate because questions that were answered outside of the day the order was placed will skew the single-day view.

This stabilizes over time, so reports over a larger time range and at least several days in the past are more accurate. But when viewing individual days, it is possible to see quite a bit of variation. So utilizing this report mitigates this particular edge case and allows you to take a more stable look at your daily data.