Integration Architecture

Integration Observability Working Skill

Build monitoring that catches integration failures before business impact, and create diagnostic runbooks so the first responder knows exactly what to check.

What this skill is for

This skill helps you map each integration to its business impact, define failure modes and detection methods, set alert thresholds, assign alert recipients, create diagnostic runbooks, and validate the entire monitoring setup with simulated failures.

When to use this skill

A new integration is going live and monitoring is "to be defined later."
Business users report missing data before any technical alert fires.
Middleware logs show errors but no one reviews them until a business escalation.
An AMS handover requires operational documentation for existing integrations.
A middleware or platform upgrade changes logging and alerting behavior.
SLAs are being defined and you need to prove they can be monitored.

Real work situations

Situation 1: Customer orders stop flowing to the warehouse

Orders from the e-commerce platform to SAP S/4 are processed successfully, but the warehouse file transfer from SAP to the logistics system stops. No alert fires because the file transfer job does not have monitoring. The issue is discovered when picking lists do not print the next morning. Warehouse operations lose half a day.

Situation 2: Middleware logs show 500 errors but no one acts

The API gateway logs hundreds of 500 errors per day for a customer lookup API. The errors are visible in a dashboard but not alerted. The business team assumes SAP is slow. The SAP team assumes the middleware is misconfigured. No one owns the alert, so no one investigates.

Situation 3: New integration launched without monitoring

A project delivers a new billing integration on time. The go-live checklist has a monitoring item marked "post-go-live." After go-live, the project team disbands. The integration runs silently for two weeks, then fails. There is no runbook, no alert, and no one who knows how to diagnose it.

Inputs required

Interface inventory with ownership matrix.
SLA requirements: availability, latency, throughput, freshness.
Middleware monitoring capabilities: dashboards, metrics, log aggregation.
SAP monitoring tools and transactions: SM58, SMQ1, SMQ2, BD87, WE02, SXI_MONITOR, AIF.
Alerting infrastructure: email, SMS, paging, ticketing integration.
Runbook templates or existing operational documentation.
Business impact mapping: which processes fail when each interface fails.
Historical incident data: how past failures were detected and resolved.

Questions to ask

What is the business symptom when this integration fails?
How long can this integration be down before business impact occurs?
Who receives the alert, and who covers when that person is unavailable?
What is the escalation path if the first responder cannot resolve the issue?
What logs and metrics are available for this integration?
What is the mean time to detect (MTTD) versus mean time to resolve (MTTR)?
How do we distinguish between a total failure and a partial degradation?
What is the false positive rate of the current alerts, and how do we tune it?

Working method

Map each interface to business impact. For every interface, document the business process affected, the symptom of failure, and the cost or risk per hour of downtime.
Define failure modes per interface. List the ways this interface can fail: timeout, schema mismatch, auth failure, data validation error, rate limit, downstream unavailable, silent data loss.
Choose detection method per failure mode. Match the failure to a detectable signal: HTTP status code, queue depth, IDoc status, file absence, data volume anomaly, latency spike.
Set alert thresholds. Define the metric, threshold, evaluation window, and severity. Base thresholds on SLA and historical baseline, not guesswork.
Assign alert recipients. Each alert goes to a named person or team with accountability, not a broad distribution list.
Create diagnostic runbook per alert. For each alert, write: what to check first, which transactions or logs to open, common causes, escalation criteria, and safe actions.
Test alert with simulated failure. Trigger each failure mode in a controlled way. Verify the alert fires, the recipient receives it, and the runbook leads to diagnosis.
Review weekly for the first month after go-live. Tune thresholds, fix false positives, and update runbooks based on real incidents.

Decision rules

If business impact exceeds $10,000 per hour, alert within 5 minutes of failure detection.
If the failure is silent (no error code), use data volume, latency, or freshness anomaly detection.
If SAP is involved, monitor SM58 (tRFC errors), SMQ1/SMQ2 (qRFC queues), BD87/WE02 (IDoc status), and AIF errors.
If middleware is involved, monitor queue depth, error rate, latency, and consumer lag.
If an alert fires more than three times per week without action, tune the threshold or fix the root cause.
If no runbook exists for an alert, create one before the interface goes live.
If an alert goes to a distribution list with more than five people, replace it with a named owner and an escalation path.
If a file transfer is involved, monitor file arrival time, file size, and record count, not just job completion.

Deliverables

Monitoring Configuration Specification — Metrics, thresholds, evaluation windows, severities per interface.
Alert Matrix — Alert name, condition, recipient, escalation, runbook link. See template below.
Diagnostic Runbooks — One per critical alert: check steps, transactions, common causes, safe actions.
Integration Health Dashboard Definition — What to display, refresh rate, audience.

Templates

Alert Matrix

---
artifact: Alert Matrix
id: ALM-001
date: YYYY-MM-DD
scope: Integration landscape | Project | Domain
---

## Alert definitions

| Alert ID | Interface | Failure Mode | Detection Method | Metric | Threshold | Window | Severity | Recipient | Escalation | Runbook |
|----------|-----------|--------------|------------------|--------|-----------|--------|----------|-----------|------------|---------|
| AL-001 | SAP→CRM Customer API | Timeout / 5xx | HTTP status | Error rate | > 1% | 5 min | P1 | AMS Lead | Integration Mgr | RB-001 |
| AL-002 | SAP→WH IDoc | Status 51 | IDoc status | Status 51 count | > 0 | 15 min | P2 | AMS Team | SAP Dev Lead | RB-002 |
| AL-003 | Bank→SAP File | File missing | File arrival | File age | > 2h | 1h | P1 | AMS Team | Finance Mgr | RB-003 |
| AL-004 | Event: OrderConfirmed | Consumer lag | Kafka lag | Lag per partition | > 10,000 | 5 min | P2 | Integration Team | Event Owner | RB-004 |

## Runbook index

| Runbook ID | Alert IDs | First Check | Key Transactions / Logs | Common Causes | Safe Actions | Escalation Criteria |
|------------|-----------|-------------|---------------------------|---------------|--------------|---------------------|
| RB-001 | AL-001 | Middleware health dashboard | API gateway logs, SAP SM58 | Auth expiry, SAP overload | Check SM58, verify token | > 3 failures in 1h |
| RB-002 | AL-002 | BD87 / WE02 for status 51 | IDoc content, segment details | Data validation, missing mapping | Review IDoc, check AIF | > 10 IDocs stuck |
| RB-003 | AL-003 | File transfer job log | Job scheduler, file system | Network issue, job failure | Verify job status, check path | File > 4h missing |
| RB-004 | AL-004 | Kafka consumer group lag | Consumer metrics, logs | Consumer down, slow processing | Check consumer health, restart | Lag growing > 1h |

## Review log
| Date | Action | Owner |
|------|--------|-------|
| YYYY-MM-DD | Initial creation | Name |

Quality checklist

Every critical interface has at least one alert.
Every alert has a defined threshold and evaluation window.
Every alert has a named recipient with accountability.
A runbook exists for every alert that pages outside business hours.
False positive rate is under 20% after the first month.
Business impact is documented per interface.
Simulated failure test confirmed alert delivery and runbook accuracy.
Dashboard shows both technical health and business outcome indicators.

Common mistakes

Mistake: Monitoring middleware health only, not business outcome. Consequence: The queue is healthy but no messages are flowing; the alert never fires.
Mistake: Alerting on every error without threshold tuning. Consequence: Alert fatigue; real failures are buried in noise.
Mistake: No runbook, so the recipient does not know what to do. Consequence: Alerts are acknowledged and ignored, or escalated blindly.
Mistake: Alert goes to a distribution list with no accountability. Consequence: Everyone assumes someone else will handle it.
Mistake: Monitoring job completion but not file content or record count. Consequence: Empty files or truncated transfers go undetected.

Agent instructions

Map to business impact first: Before choosing technical metrics, document what business process fails and how fast.
Use SAP-specific monitoring transactions: Reference SM58, SMQ1/SMQ2, BD87, WE02, and AIF for SAP-centric integrations.
Produce an Alert Matrix and runbooks: Do not stop at a list of metrics. Deliver actionable artifacts.
Avoid generic language: Do not write "monitoring is important for reliability." Write "Alert on IDoc status 51 within 15 minutes."
Handle missing infrastructure: If alerting infrastructure is missing, list it as a dependency and block go-live until resolved.
Link to Atlas diagnostics: Reference SAP Interface Monitoring Diagnostics, SAP IDoc Status Diagnostics, and SAP qRFC/tRFC Diagnostics for specific check steps.

Related skills

Integration Error Handling — Design what happens when monitoring detects a failure.
Interface Ownership — Assign alert recipients and runbook owners.
Incident Triage — Respond when alerts fire.
Root Cause Analysis — Investigate recurring alert patterns.

Related Atlas pages

SAP Interface Monitoring Diagnostics — SAP-specific monitoring.
SAP Integration Error Handling Diagnostics — Error patterns to monitor.
SAP IDoc Status Diagnostics — IDoc failure detection.
SAP qRFC / tRFC Diagnostics — Queue monitoring.
SAP Outbound Processing Diagnostics — Outbound failure patterns.
SAP Inbound Processing Diagnostics — Inbound failure patterns.

Verification status and limitations

This skill is a public working interpretation of integration observability practice. It is not official SAP, Datadog, Splunk, or vendor documentation. SAP-specific monitoring references standard transactions available in common S/4HANA and ECC releases; custom landscapes may have additional or different tools. The skill assumes some form of middleware or platform monitoring exists; if none exists, the first step is infrastructure procurement, not configuration.