Summary

Orchestration Engine is designed to give you, the communicator, the freedom to focus on your message and still reach the right employees through their preferred channels. We have designed the engine to reduce noise, increase engagement, and prioritize notifications to drive action.

When you set the Orchestration Engine to “optimize”, the engine will determine when and how to reach a user via notification. The individualized notification plan is based on campaign configuration (especially priority and duration), a user’s current campaign queue, and the user’s history of engagement. In the absence of data about a single user, we’ll rely on community and platform history of engagement.

We provide real-time feedback about the overall effectiveness of your campaign through the campaign-level scores, performance, and fatigue. You can improve these scores by altering your priority, duration, and other settings on the Orchestrate tab.

Engines

Optimize

The "optimize" engine decides everything for you. For each member of the target audience, it determines the best time and channel to reach them. It also determines the best time and channel to re-engage with the user (if the user has not already interacted with content).

Hybrid

The "hybrid" engine" allows you to schedule the date and time of the first communication. The engine will then determine the best time and channel to re-engage with the user.

Override

You can use the "override" engine to schedule the date and time of a communication. "Override" will not attempt to re-engage with the user and a duration cannot be selected, as this is a one-time communication attempt. If a member of the audience chooses not to view the campaign, the engine will not attempt to engage them again. With "override", if you select one channel (email, assistant, or push notification), the engine will only publish to the audience via the selected channel. If multiple channels are selected, the engine will determine the best channel to reach each member of the audience on. "Override" should be used sparingly in order to allow other campaigns to optimize efficiently.

How Does the “Optimize” Engine Determine How to Reach a User?

To leverage the full power of the Orchestration Engine, we recommend keeping the engine selection set to “optimize”. When enabled, the optimization setting will evaluate all the possible campaign plans for an individual within the configured duration. Each possible plan is given a score, and we implement the highest-scoring plan per user.

Because each notification delivery plan is individualized, different users may see notifications in different channels, at different times, on different days, and receive a different number of follow-up reminders. This is exactly what you want - to reach your users when and how they prefer to be engaged.

How, exactly, do we score campaign plans? We score all possible campaign plans based on cost, value, and likelihood of engagement:

• Cost - Some channels are more disruptive than others, so we weigh them in this order of low to high cost: Assistant, email, push. Also, on a given day, each additional communication is more expensive. To account for the increased expense of multiple notifications, cost takes into account how many other campaigns are already scheduled for the user to receive.
• Value - We look at both the priority and how new the campaign is - we assume it’s more important to receive a notification shortly after publishing. Value is impacted by your campaign configuration.
• Likelihood - We weigh the likelihood of an individual user engaging via Assistant, email, or push at any given hour of the day. In the absence of individual data, we make assumptions based on community or platform history.

Note: It takes about 5 days of solid engagement data to really personalize the likelihood score for a user. This data is evaluated week by week, so if habits change (say, during travel), it could take up to a week to shift.

Reengagement

When set to Optimize or Hybrid, the Orchestration engine will attempt re-engagement with users within the assigned Duration. Reengagement attempts will end once the user has seen the campaign (through an email open, push open, or viewing the campaign directly in a feed).

Note, the reengagement plan for an individual user is reevaluated frequently during the campaign duration. Publishing a campaign sets the plan into motion. Once a campaign is scheduled and published, a replan will be triggered every 24 hours, at minimum. Additional events can trigger a replan (see below). Replanning ends when the campaign duration ends.

Events That Trigger a Replan

1. Communication update - This includes any change to an individual campaign, as well as the addition of another campaign to a user's campaign queue.
2. Audience update - Updates to the campaign's target audience will trigger a re-plan.
3. User interaction - User interaction includes campaign views and engagements such as sharing and acknowledgment. 
4. Replanner job - Replanner jobs run every 6 hours for all users whose plans have not been updated in the last 24 hours.
   

What Do Performance and Fatigue Mean?

Performance

Performance is a measure of the overall likelihood of a particular audience engaging given a particular campaign configuration. The higher the performance score the more people we predict will engage with your message. There's always a chance the end of campaign metrics will show better or worse performance than the score initially indicated.

Aim for a high-performance score. The Orchestration Engine in optimize mode will maximize performance for a campaign by selecting the best possible plan for each user given the campaign settings. If you select “hybrid” or “override”, you may see performance drop as you customize the settings. For example, if a sizable chunk of the audience has a higher likelihood of opening push notifications than emails and you turn off push notifications as a possible channel, the performance score will likely drop.

Fatigue

Noise averages the cost of a campaign and takes into account reachability. Some audiences are harder to get in touch with than others. We determine this via a reachability score based on how many users have an email, have downloaded the app, etc. If a user is reachable via one channel only, this makes it far more likely that they feel overwhelmed by notifications on that channel.

Aim for a low fatigue score. The Orchestration Engine in “optimize” mode will minimize noise for a campaign by selecting the best possible plan for each user given the campaign settings. If you select “hybrid” or “override”, you may see fatigue increase as you customize the settings. If you narrow down the delivery time and target channels, you may increase the fatigue of your campaign by colliding with other scheduled campaigns or by preventing the engine from potentially choosing a less costly channel, even if it's more efficient for a particular user.

Note: We take a small sample of the audience in order to generate the channel insights.

How Does the Orchestration Engine Work with Email-Only?

In general, for Email-Only, there is no change to how the Orchestration engine works, only a restriction of which channels it evaluates as ways to reach the member. The engine will only evaluate email as a reachable channel. This places more emphasis on the other variables - such as how many emails the member is already receiving on a given day, what the other campaigns are, etc.

Frequently Asked Questions

Where can I see the orchestrated campaigns?

On the Orchestrate tab, there is a hover to explain competing campaigns.

Why do some users receive multiple emails at one time?

Delivering at the same time prioritizes our goal of engaging users for “2 minutes a day”. You can also think of this as minimizing randomization and enabling focused work.

Note: We have considered the value of “batching” these emails into one email. However, if we did batch the emails, we would have to impose our own template to the content - likely having to link to the full content. You would not get the ‘what you see is what you get’ experience of composing the email in Studio and the recipient seeing exactly what you created.

What do I do if I have time-sensitive content?

You can use the duration setting to mark the end of when the campaign would be relevant, and you can schedule archiving if the campaign truly needs to be stopped at a certain date and time.

What do I do if I have an urgent communication that MUST go out via email? (or other channel)

If you need to know that a particular communication has occurred at a specific time, then you should use the Override or Hybrid setting. Just navigate to Orchestrate > Rules and choose Override or Hybrid. Then you can select or deselect the targeted channels as necessary. Reminder: Using "override" means that the system will not send re-notifications. 

Note: keeping the engine set to Optimize while setting a campaign to Critical with short duration does NOT guarantee that communications will go out at a certain time to the recipients. Once Optimize is engaged, the engine determines whether or when to send a notification based on the recipient's queue. If there are a lot of "critical" communications competing for attention, some may not qualify for a notification to that particular user.

How do I cap re-notifications? Why do some posts get a lot of re-attempts and others get very few?

We re-attempt notifications if the user has not seen the content. This is true of all campaigns, including ones marked as needing “acknowledge”.

Regarding how many re-attempts, that number is based on both the campaign value and their personal queue. High-priority campaigns get more notifications than low-priority campaigns. Quiet weeks for the user may see more re-notifications than busy weeks when there is a lot of new content competing for “first notification”.