Skill Hub — DAMA / Data

Data Integration & Interoperability Working Skill

Diagnose integration failures caused by data mismatch, schema drift, or mapping errors. Define data contracts and validation rules at interface boundaries.

What this skill is for

This skill helps you fix the data layer of integration failures. It provides a method to diagnose why data does not flow correctly between systems: schema mismatches, field mapping errors, validation gaps, reference data drift, and timing issues. The output is a corrected interface specification with a data contract, validation rules, and monitoring that prevents recurrence.

When to use this skill

An integration fails and the error is in the data payload: missing fields, wrong format, or unexpected values.
A source system sends data that the target system rejects or misinterprets.
An API schema changes and downstream consumers break.
Data arrives at the target system but is stored in the wrong field or table.
An IDoc, message, or file transfer succeeds technically but produces wrong business results.
A new interface is being designed and needs a data contract before development starts.

Real work situations

Example 1: API schema drift breaks mobile app

A backend API adds a new required field region_code without versioning. The mobile app does not send it. The API rejects requests with a generic 400 error. The skill diagnoses the schema change, defines a backward-compatible versioning rule, and produces a data contract that specifies required vs optional fields and notification rules for schema changes.

Example 2: IDoc field mapped to wrong target

A customer master IDoc from SAP to CRM maps STCD1 (tax number) to the CRM "VAT" field. After a system upgrade, the mapping silently changes to "Company Registration Number." Tax reports in CRM are now wrong. The skill traces the mapping, identifies the change point, and produces a mapping validation test and a change notification rule.

Example 3: File load with wrong delimiter encoding

A daily CSV file load from a supplier shifts all columns by one position because the file switches from comma-delimited to semicolon-delimited. The load succeeds but product descriptions are stored in the price field. The skill defines a file format contract with validation rules: delimiter, encoding, header row, and column count check.

Example 4: Real-time event payload too large

An event-driven architecture sends order events to a warehouse system. During peak hours, the payload exceeds the message broker limit. Events are dropped silently. The skill defines a payload size contract, a splitting rule for large orders, and a monitoring alert for dropped messages.

Inputs required

The interface identifier: API endpoint, IDoc type, message queue, file transfer, or integration flow name.
Source system schema: fields, types, lengths, formats, and validation rules.
Target system schema: expected fields, types, lengths, and constraints.
Mapping documentation or configuration showing how source fields map to target fields.
Error logs, failed payloads, or incident tickets showing the failure pattern.
Business process description of what the integration is supposed to achieve.
Integration ownership: technical owner, business owner, and operational support contact.

Questions to ask

What is the exact data element that fails, and what is the expected vs actual value?
Which field in the source maps to which field in the target, and is the mapping still correct?
What validation does the target apply, and does the source data meet it?
Has the source schema, target schema, or mapping changed recently?
What reference data or code values are involved, and are they synchronized?
What is the payload size, format, encoding, and frequency? Are any limits being exceeded?
What happens to failed data: is it rejected, quarantined, retried, or lost?
Who is notified when the interface fails, and how long does it take to respond?

Working method

Document the interface. Record: interface ID, source system, target system, direction, data domain, business purpose, and owners. Use the Interface Ownership Matrix template.
Capture the failure symptom. Record: error message, failed payload sample, timestamp, frequency, and business impact. Be exact.
Compare source and target schemas. List source fields with types, lengths, and formats. List target fields with constraints. Identify mismatches: missing fields, type conflicts, length overflows, and format errors.
Verify the mapping. Trace each source field to its target field. Check for: missing mappings, wrong mappings, stale mappings, and silent truncation.
Check reference data. Verify that code values, status values, and identifiers used in the payload are valid in the target system.
Check timing and sequencing. Verify that data arrives in the correct order, at the correct frequency, and within time windows that the target expects.
Classify the failure type. Use: schema mismatch, mapping error, validation failure, reference data mismatch, format error, size limit, timing issue, or unknown.
Design the fix. Choose: correct mapping, add validation, synchronize reference data, change format, split payload, add sequencing, or redesign the interface.
Define the data contract. Document: schema version, required and optional fields, types, lengths, formats, allowed values, payload limits, frequency, error handling, and change notification rules.
Define monitoring and alerting. Design checks for: schema compliance, mapping integrity, reference data sync, payload size, error rate, and latency. Define thresholds and alert owners.
Produce the Integration Failure Review. Document symptom, classification, fix, contract, and monitoring. Use the template.
Validate with a test transaction. Run a controlled test through the fixed interface. Verify that the data arrives correctly and that monitoring catches intentional errors.

Decision rules

If the source schema changes, the data contract must be versioned and consumers must be notified before the change is deployed.
If a field is mapped incorrectly, fix the mapping before correcting data in the target; otherwise the next load will reintroduce the error.
If reference data mismatch causes the failure, synchronize codes before reprocessing payloads.
If the payload exceeds size limits, split the payload or increase the limit; do not silently truncate data.
If validation fails at the target, add validation at the source or in the integration layer; do not weaken target validation.
If an interface has no owner, assign one before designing fixes; unmaintained interfaces break again.
If the failure type is unknown, add logging and payload capture before the next occurrence; do not guess.

Deliverables

Interface Ownership Matrix entry — documented interface with owners and SLA. See Artifact Templates.
Integration Failure Review — classified failure, root cause, fix, and monitoring. See Artifact Templates.
Data Contract — schema, validation, format, frequency, and change rules.
Mapping Validation Test — automated or manual test that verifies source-to-target mapping.

Templates

Data Contract (compact)

---
contract: Data Interface
id: DC-001
interface: Customer Master to CRM
version: 2.1
status: active
---

## Schema
| Source Field | Source Type | Target Field | Target Type | Required | Validation |
|--------------|-------------|--------------|-------------|----------|------------|
| KUNNR | CHAR(10) | customer_id | VARCHAR(20) | Yes | Numeric, leading zeros preserved |
| NAME1 | CHAR(40) | name | VARCHAR(100) | Yes | Trim trailing spaces |
| STCD1 | CHAR(20) | tax_number | VARCHAR(20) | Conditional | Required if country = DE |
| LAND1 | CHAR(3) | country_code | CHAR(2) | Yes | Must exist in target country list |

## Format
- Payload: JSON
- Encoding: UTF-8
- Max size: 512 KB
- Array max: 100 customers per message

## Frequency
- Real-time for changes
- Full sync: daily at 02:00 UTC

## Error handling
- Schema violation: reject with 400, log error, alert integration owner
- Reference data mismatch: quarantine payload, alert data steward
- Size limit exceeded: split payload, alert technical owner

## Change process
- Minor changes: notify consumers 5 business days in advance
- Major changes: version bump, consumer migration required
- Notification: email to interface owners + wiki update

Mapping Validation Checklist

| Check | Method | Owner | Frequency | Last Run | Status |
|-------|--------|-------|-----------|----------|--------|
| All source fields mapped | Compare source schema to mapping doc | Integration Lead | Monthly | 2026-06-01 | Pass |
| Target constraints satisfied | Test payload against target validation | QA | Weekly | 2026-06-08 | Pass |
| Reference data sync | Compare code lists in source and target | Data Steward | Daily | 2026-06-09 | Fail — 3 missing codes |
| Payload size within limit | Monitor message broker metrics | Ops | Continuous | 2026-06-09 | Pass |

Quality checklist

The interface has a named technical owner, business owner, and operational owner.
Source and target schemas are documented and compared.
Every source field is mapped to a target field or explicitly marked as unmapped.
The failure is classified into a specific type, not described as "integration error."
The fix addresses the root cause, not just the symptom.
A data contract exists with schema, validation, format, frequency, and change rules.
Monitoring checks schema, mapping, reference data, size, and error rate.
A test transaction has validated the fix and the monitoring.

Common mistakes

Mistake: Fixing the target data without fixing the mapping or source. Consequence: The next integration run reintroduces the same error; the fix becomes a recurring manual task.
Mistake: Weakening target validation to accept bad data. Consequence: The target system becomes polluted; downstream reports and processes fail.
Mistake: Changing a schema without versioning or notification. Consequence: All consumers break simultaneously; incident volume spikes.
Mistake: Ignoring payload size and frequency limits. Consequence: Silent message loss during peak hours; data gaps are discovered days later.
Mistake: Documenting the interface in a wiki with no link to the actual code or configuration. Consequence: Documentation drifts from reality; troubleshooting takes longer.

Agent instructions

When using this skill, an AI agent must:

Start with the exact failure. Ask for: interface ID, error message, payload sample, and timestamp. Do not proceed with "the integration is not working."
Compare schemas explicitly. List source fields and target fields side by side. Identify mismatches. Do not assume the mapping is correct.
Check reference data. If codes or identifiers are involved, verify they exist in the target. Reference data mismatch is a common hidden cause.
Classify the failure type. Use the defined types. If none fit, say "unknown" and propose logging improvements.
Produce a data contract. Even for existing interfaces, document the contract that should have existed. This prevents future drift.
Produce artifacts, not advice. Output an Integration Failure Review and a Data Contract in the templates provided.
Link to Atlas diagnostics. If the integration involves SAP, link to SAP IDoc Status Diagnostics, SAP Business Partner Replication Diagnostics, or SAP CVI Synchronization Diagnostics.
Handle missing information. If schema documentation or payload samples are unavailable, produce a collection checklist and ask the user to gather them.

Related skills

Reference Data Management — when code values cause integration failures.
Data Quality Root Cause — when data defects need deeper tracing.
Data Lineage — when the data path across systems is unclear.
Interface Ownership — when ownership gaps block resolution.
Integration Error Handling — when error handling patterns need design.

Related Atlas pages

SAP IDoc Status Diagnostics — classifying IDoc failures.
SAP Business Partner Replication Diagnostics — BP replication data issues.
SAP CVI Synchronization Diagnostics — customer-vendor integration data issues.
Data Contracts — formal agreements at interface boundaries.

Verification status and limitations

This skill is a public working interpretation of data integration and interoperability practice. It is not official DAMA-DMBOK, SAP, or API design documentation. It has been applied in SAP-centric and API-centric integration contexts but may need adaptation for event-driven architectures, streaming platforms, or microservices mesh.

Limitations: This skill does not cover advanced API gateway configuration, message broker tuning, or cloud integration platform administration. It focuses on data-layer diagnosis, contract design, and validation suitable for integration analysts and data stewards.