The Results page provides one unified view of delivery and engagement metrics for your campaigns, including metrics like dropped messages and delivery errors. The Results page also allows you to drill-down to Insights reports such as the Campaign Performance, Campaign Delivery, Poll Performance reports, etc. via embedded links.

Note: The Results page will only appear for campaigns that are published, scheduled, or archived in the new Studio. The Results page will not appear for Classic Studio campaigns.

Results Page Expectations

The Results page is meant to surface campaign Delivery information to you, even ahead of the data populating in the Insights reports (Insights reports typically have an expected latency of 1-2 hours). This means that it is expected for the Results page and the linked Insights reports on the Results page to be out of sync immediately after publish, and for some time within the campaign’s lifecycle. The Delivery metrics on the Results page should always be ahead of the Insights reports. The Engagement metrics section will most likely be the same, or a little behind the Insights reports, depending on the point in the cache strategy.

Additionally, you should be aware that the metrics in the Engagement and Delivery sections are updating on different schedules as well. As a general rule, the Delivery section should be close to real-time accuracy, though it is possible that the most up-to-date info could take up to a maximum of 1 hour to populate. This latency limit can be driven by a number of factors, including the size of the audience of the campaign, i.e. large campaigns will take longer to have data populated. The Engagement section of the Results page could take a maximum of 4 hours to be up-to-date, due to our caching strategies.

Results Page Access

The Results page can be accessed from the Campaigns page by clicking on the campaign or selecting View Results from the menu after clicking the ellipses to the right of the campaign. It can also be accessed by clicking on the campaign on the Calendar page. Any user who has the ability to view the campaign will also be able to view the Results page, however, they will not be able to navigate to Insights when they click the report links if they do not have permissions for Insights.

Results Page Details

At the top of the Results page, you will see the campaign title, as well as options to edit, duplicate, or archive the campaign.

Campaign Status

The top box will show you the status of your campaign. If complete, the box will state 'Complete'. If not, the remaining number of days will be displayed instead.

Engagement

The Engagement section provides insight into how members are engaging with the campaign. Clicking on the View Full Report button will take you to the Campaign Performance report for the campaign.

Note: For a full list of our metric terms, check out our Glossary of Terms.

Embedded Media

The Embedded Media section is reserved for video and poll content. Clicking on the View button will take you to the associated Insights report for that piece of content: the Poll Performance report or Video Performance report.

Delivery

The Delivery section provides insight into the campaign delivery details. Clicking on the View Full Report button will take you to the Campaign Delivery report for the campaign.

Note: Messages refer to all email, push notifications, and Assistant notifications.

Plan Panel

Post-publish, the Plan panel in Deliver will default to a new view, listing the time remaining in a campaign's duration, and a button to view the Campaign Delivery Details report in Insights.



Clicking the View Remaining Prediction button will display the full Plan panel with the updated campaign prediction.

What Do Performance and Fatigue Mean?

Performance and Fatigue give real-time feedback on the overall predictive performance of your campaign when creating or editing a campaign. These are especially relevant if you choose to only enable Automatic Retargeting or disable Engagement Boost, which does not make any attempts to choose the optimal messaging settings for your users.

Performance

Performance is a measure of the overall likelihood of a particular audience engaging given a particular campaign configuration. The higher the performance score the more people we predict will engage with your message. There's always a chance the end-of-campaign metrics will show better or worse performance than the score initially indicated.

Aim for a high-performance score. Enabling both Optimized Delivery and Automatic Retargeting will maximize performance for a campaign by selecting the best possible plan for each user given the campaign settings. If you only select Automatic Retargeting or disable Engagement Boost, you may see performance drop as you customize the settings. For example, if a sizable chunk of the audience has a higher likelihood of opening push notifications than emails and you turn off push notifications as a possible channel, the performance score will likely drop.

Fatigue

Fatigue is a measure of information overload. Fatigue averages the cost of a campaign and takes into account reachability. Some audiences are harder to get in touch with than others. We determine this via a reachability score based on how many users have an email, have downloaded the app, etc. If a user is reachable via one channel only, this makes it far more likely that they feel overwhelmed by notifications on that channel.

Aim for a low fatigue score. Enabling both Optimized Delivery and Automatic Retargeting will minimize noise for a campaign by selecting the best possible plan for each user given the campaign settings. If you only select Automatic Retargeting or disable Engagement Boost, you may see fatigue increase as you customize the settings. If you narrow down the delivery time and target channels, you may increase the fatigue of your campaign by colliding with other scheduled campaigns or by preventing the engine from potentially choosing a less costly channel, even if it's more efficient for a particular user.

If an individual user has many campaigns competing for their attention, to minimize fatigue the engine will send notifications for the highest scoring campaigns (priority, recently published, etc.) and some campaigns may not be sent as a notification and instead rely on the user seeing them in the member experience.

Note: We take a small sample of the audience in order to generate channel insights.

FAQ

Should the Results page appear for every campaign that is published?

The Results page will only appear for Studio campaigns that have an audience/deliveries, or campaigns that had an audience at some point within their lifecycle. For example, a campaign with an audience that is updated to a topic-only campaign. The Results page will appear for sent, previously published but now archived campaigns, and scheduled campaigns. It will not appear for campaigns that are in draft or need review.

Is the Successful Delivery count unique?

No, these counts include delivery counts per user channel, and include both initial deliveries and retargeting attempts. Ex: A campaign that is sent to a user that delivers an email and an Assistant notification on day one, and then a retargeting via email on day 3, will count as 3 successful deliveries. This means that the channel counts are all-inclusive as well.

Does the Dropped/Errors count only apply to email?

No, those counts will apply to both email and push notifications.

What is the difference between Dropped and Errors?

• Dropped - The total number of messages that have been dropped. A drop occurs if the user is missing from the audience or has been unsubscribed from the community. The engine will not attempt deliveries for users who are removed from the audience or the community, or unsubscribed from the email campaign itself. Note: Drops are not errors — drops can happen if the user was blocked at the moment the message was sent.
• Errors - The total number of failed sends and/or deliveries. Errors can occur when trying to attempt a send or can occur when a send event has been initiated, but delivery cannot be confirmed. Both are counted in this metric.
  • ERRORS = SENT_ERROR + DELIVERY_ERROR
  • SENT_ERROR - An error trying to send the message
  • DELIVERY_ERROR - An error trying to finalize delivery of the message

Please see this article in relation to why an email may fail to be delivered:

Will the Results page show all metrics associated with the campaign, or only the latest version of the campaign?

The Results page will show all metrics associated with a campaign, so a campaign that was initially delivered via email to users and then made a topic-only campaign later on would still depict those email deliveries in the Results page.

Should these metrics match the Insights reports and the hover-over metrics on the Campaigns page?

The Results page should always match the campaign hover-over. A delay between the Results page and the Insights reports is expected, but eventually all the counts should align. 

Will users without permission to the Insights reports see the report buttons on the Results page?

No, these users will not have the report buttons on the Results page or on the Plan panel in Deliver.