Building our own monitoring for Adobe Journey Optimizer

When Adobe announced the end of support for Adobe Campaign v7 by March 2027, the decision on our side was easy. We were already running on Adobe Experience Platform, so moving our mailing activation to Adobe Journey Optimizer wasn’t really a question of “if,” it was a question of “how.” The migration of the mailings themselves was hard enough on its own. But monitoring turned out to be a different kind of hard: it wasn’t something we could just switch on. It had to be conceived and architected from scratch

Adobe Campaign gave us years of operational muscle memory: delivery reports, bounce logs, error dashboards, all built in. AJO isn’t blind, it comes with standard reporting for the usual suspects: sent, fails, bounces. That reporting runs through the CJA reporting UI embedded directly in AJO, the default view, not a separate CJA connection. We deliberately haven’t connected AJO data into CJA yet, because that would count against our licensed rows, and for standard campaign and journey-level reporting we don’t need to (at the moment).

But that default reporting has one hard limit that mattered a lot for us: there’s no alerting. It’ll tell you how a campaign or journey performed, and it does give you visibility into individual nodes inside a journey, but there’s no alerting system sitting on top of any of it. Nothing triggers if a specific node fails, if bounce rates spike, or if a delivery stalls. You can look at the numbers, but nothing taps you on the shoulder when they go wrong. If you’re moving your entire mailing activation to a new tool, that gap is not a nice-to-have. It’s a blocker.

So we built the monitoring and alerting ourselves.

The problem with moving to AJO

Every mailing platform migration has the same underlying risk: something breaks silently, and nobody notices until a business stakeholder asks why open rates dropped or why a campaign didn’t go out at all. In Adobe Campaign, we relied on built-in delivery monitoring that would flag problems as they happened. Standard AJO reporting can tell you the same core numbers, sent, fails, bounces, but only at the campaign or journey level, and only if someone goes and looks at it. It has no concept of “notify me.”

That’s not a criticism of AJO as a product. It’s built around AEP’s data model, and AEP is generous with data but opinionated about how you access it. Everything about your mailings, errors, bounces, delays, and anything more granular than the standard reports, like the behavior of a specific journey node, ends up in datasets on the platform. It’s there. It’s just not surfaced anywhere you’d actually look during a migration cutover, and even the parts that are surfaced don’t push a notification to anyone.

We couldn’t accept “the data exists somewhere” as an answer. We needed active monitoring: something that would tell us when a threshold was crossed, not something we’d have to remember to check.

The stack we built

We ended up with four components, each doing exactly one job:

LayerToolRole
Data accessAdobe Query ServicePull the metrics we need out of AEP datasets
ExposureAdobe App BuilderRun the queries on a schedule and expose results via API
Metrics & alertingPrometheusTurn API responses into metrics, persist them, evaluate thresholds
VisualizationGrafanaDashboard the metrics for the team
Table 1: Techstack

None of these tools were chosen because they’re trendy. Query Service was the only realistic way to get structured metrics out of our AEP datasets. App Builder was the natural bridge because it’s already part of our AEP tooling and can be scheduled and exposed as an API without standing up separate infrastructure. Prometheus and Grafana are the standard combination for anyone who’s done ops monitoring before, and reusing that pattern meant we didn’t have to reinvent alerting logic from scratch.

Figure 1: 4 Layer Blueprint

1. Query Service: Getting the metrics out

All the mailing-relevant data, errors, bounces, delays, and the rest, lives in datasets in AEP. Query Service is how we get it out in a usable shape.

Instead of writing one-off queries per metric, we standardized on a query template with the same output structure every time:

  • datasetName
  • metricName
  • value
  • exportTimestamp

Every query we write follows that shape, no exceptions. That consistency is what makes the rest of the pipeline dumb and reliable: App Builder doesn’t need to know anything about the specific metric it’s fetching, it just needs to know the shape of the result.

We also gave every query the same naming prefix, which made them trivially discoverable later. That decision looks small, but it’s the difference between App Builder doing simple template discovery and us having to hardcode a list of queries somewhere that inevitably goes stale.

Here’s roughly what a template looks like in practice:

SELECT
'ajo_message_feedback_event_dataset' AS datasetname,
_experience.customerJourneyManagement.messageDeliveryfeedback.feedbackStatus AS metricname, COUNT(1) AS metricvalue, DATE_SUB(CURRENT_DATE, 1) AS exportdate
FROM cjm_message_feedback_event_dataset
WHERE DATE(timestamp) = DATE_SUB(CURRENT_DATE, 1)
GROUP BY _experience.customerJourneyManagement.messageDeliveryfeedback.feedbackStatus;

2. App Builder: Triggering and exposing

Adobe App Builder does two jobs in this pipeline. First, it retrieves all query templates via the templates API endpoint, filtered by our naming prefix. Second, it triggers each of them and exposes the results through its own API endpoint.

That prefix convention is what makes the pipeline extensible without touching App Builder’s code at all. Add a new query template with the same prefix, and it’s automatically picked up on the next run, no deployment, no config change, nothing to wire up manually.

This is the part that actually solves the “Adobe doesn’t offer monitoring” problem. App Builder becomes the bridge between “data exists somewhere in AEP” and “data is available as a clean, anonymous API response.” Everything exposed here is aggregate metric data, no PII, just dataset name, metric name, value, and timestamp. That matters both for compliance and for keeping the whole thing simple: there’s no identity data to protect downstream, which meant we could keep Prometheus and Grafana access relatively lightweight.

3. Prometheus: Metrics and alerting

A cron job in Prometheus polls the App Builder endpoint on a schedule, pulls in the exposed metrics, and persists them. This is where the templates and their consistent structure pay off again: Prometheus doesn’t care what the underlying business metric is, it just ingests metricName / value pairs on a timestamp and stores them as time series.

Alerting thresholds are defined directly in Prometheus against these metrics.
When a bounce rate crosses a defined threshold, or an error metric spikes, or a delay metric shows a mailing stalling, Prometheus is what notices first.

4. Grafana: Visualization

Grafana’s job here is deliberately narrow. It reads the metrics Prometheus has already collected and persisted, and turns them into dashboards. It doesn’t touch AEP, it doesn’t touch App Builder, it just visualizes what’s already been captured and evaluated upstream. Keeping Grafana this dumb was intentional: it means the dashboard layer can change or be extended without touching how the data is collected or how alerting logic works.

Why this architecture holds up

The reason this pipeline works is that each layer only does one thing, and the query template convention is what glues them together without any layer needing to understand the others’ internals. Query Service doesn’t know about Prometheus. App Builder doesn’t know what a bounce rate means. Prometheus doesn’t know it’s dealing with mailing data at all, it just sees metrics. That separation is what let us add new metrics later just by writing a new query with the same four-field structure and the same prefix, no changes needed anywhere downstream.

It’s also worth saying plainly: none of this is AJO-specific. The same pattern works for any AEP-based process where the data lands in a dataset and Adobe doesn’t give you a native alerting path. Mailing monitoring was our first use case, not our last.

Key takeaways

  • Adobe Journey Optimizer does have standard reporting for sent, fails, and bounces via the embedded CJA UI, but it’s campaign- and journey-level only, and it has no native alerting into email or Slack. If you need generic measures, node-level visibility, or any kind of notification, budget for building this yourself.
  • All the operational data you need, errors, bounces, delays, already lives in AEP datasets. The challenge isn’t availability, it’s exposure.
  • Standardizing query templates around a fixed structure (datasetName, metricName, value, exportTimestamp) and a shared naming prefix is what makes the whole pipeline extensible without custom glue code per metric.
  • App Builder is a solid bridge for exposing AEP data as an anonymous, PII-free API, which simplifies everything downstream.
  • Prometheus and Grafana aren’t AEP-native tools, but they’re the right tools for the job: proven alerting logic and visualization, without having to build either from scratch.

In