API Integration Best Practices for Modern Businesses (The Hard-Won Lessons)
Avoid costly integration mistakes with proven API best practices for reliability, security, scalability, and long-term business success.
I've been on the wrong end of a bad API integration enough times that I've started keeping a mental list of "things I wish the documentation had told me." Not the happy path stuff, that's always covered. The edge cases. The failure modes. The design decisions that seem fine until 3 AM when production is down.
This is that list, structured into something hopefully more useful than a changelog.
Start with error handling, not the happy path
This is the single biggest mindset shift that separates API integrations that hold up in production from the ones that break quietly and cause data integrity issues for weeks.
The documentation always shows you the success case:
// "Here's what you'll get when everything works"
{
"status": "success",
"data": {
"id": "cust_12345",
"email": "user@example.com"
}
}
What it spends less time on: the partial failure, the timeout that returns a 200 with an error body, the rate limit that returns a 429 with a Retry-After header you should be parsing, the webhook that fires twice for the same event.
Good API integration code handles failure first:
async function callExternalAPI(endpoint, payload) {
try {
const response = await fetch(endpoint, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': Bearer ${process.env.API_KEY},
},
body: JSON.stringify(payload),
signal: AbortSignal.timeout(5000), // Don't wait forever
});
if (response.status === 429) {
const retryAfter = response.headers.get('Retry-After') || '60';
throw new RateLimitError(`Rate limited. Retry after ${retryAfter}s`, parseInt(retryAfter));
}
if (!response.ok) {
const errorBody = await response.json().catch(() => ({}));
throw new APIError(`API error ${response.status}`, response.status, errorBody);
}
return await response.json();
} catch (err) {
if (err instanceof APIError || err instanceof RateLimitError) throw err;
// Network errors, timeouts, JSON parse failures
throw new IntegrationErrorNetwork or parse failure: ${err.message});
}
}
Typed, differentiated errors. You want to handle a rate limit differently from a server error, and both differently from a network timeout. If everything is a generic catch (err) { console.error(err) }, you'll never build proper retry logic.
Implement exponential backoff properly, most implementations get this wrong
Retrying on failure is table stakes. But naive retry logic can make things worse, hammering a struggling API with immediate retries just adds to the load.
Exponential backoff with jitter is the pattern:
async function withRetry(fn, options = {}) {
const {
maxAttempts = 3,
baseDelay = 1000,
maxDelay = 30000,
} = options;
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
try {
return await fn();
} catch (err) {
const isLastAttempt = attempt === maxAttempts;
const isRetryable = err instanceof RateLimitError ||
(err instanceof APIError && err.status >= 500);
if (isLastAttempt || !isRetryable) throw err;
// Exponential backoff + jitter prevents thundering herd
const exponentialDelay = Math.min(baseDelay * Math.pow(2, attempt - 1), maxDelay);
const jitter = Math.random() 0.3 exponentialDelay; // ±30% jitter
const delay = Math.floor(exponentialDelay + jitter);
console.logAttempt \({attempt} failed, retrying in \){delay}ms);
await new Promise(resolve => setTimeout(resolve, delay));
}
}
}
// Usage
const result = await withRetry(() => callExternalAPI('/endpoint', payload), {
maxAttempts: 4,
baseDelay: 500,
});
The jitter is important. Without it, if you have multiple workers all failing simultaneously and retrying at the same intervals, you get a thundering herd, all workers retrying at exactly the same moment, creating a new spike that fails again. Jitter distributes the retries.
Idempotency keys: use them for anything that writes data
This is the one that causes the most painful production incidents. A payment API call times out. You don't know if it went through. Do you retry? If you do and it did go through, you've just double charged someone.
The solution is idempotency keys, a unique identifier you send with each request that tells the API "If you've already processed a request with this key, don't process it again, just return the original result."
const { v4: uuidv4 } = require('uuid');
async function createPayment(customerId, amount) {
// Generate once, store alongside the transaction record
const idempotencyKey = uuidv4();
// Store the pending transaction with the key before calling the API
await db.transactions.create({
customerId,
amount,
idempotencyKey,
status: 'pending',
});
const result = await withRetry(() =>
callExternalAPI('/payments', {
customer_id: customerId,
amount,
}, {
headers: {
'Idempotency-Key': idempotencyKey, // Stripe, Braintree, many others support this
}
})
);
await db.transactions.update({ idempotencyKey }, { status: 'completed', externalId: result.id });
return result;
}
If the retry fires and the first attempt actually succeeds, the API returns the original response rather than processing again. You get a clean result, no duplicate charge.
Not every API supports idempotency keys, check the docs. But for Stripe, Square, Twilio, and many other payment/communication APIs, it's supported and you should always use it.
Webhook validation: trust nothing that arrives at your endpoint
If you're receiving webhooks and most modern API integrations involve them, you should be validating every incoming payload against a signature. Not doing this means any actor can POST arbitrary data to your webhook endpoint and trigger your handlers.
The pattern is consistent across most providers:
// Express webhook handler example (Stripe pattern)
app.post('/webhooks/stripe', express.raw({ type: 'application/json' }), (req, res) => {
const sig = req.headers['stripe-signature'];
const webhookSecret = process.env.STRIPE_WEBHOOK_SECRET;
let event;
try {
// Stripe's SDK handles HMAC-SHA256 signature verification event = stripe.webhooks.constructEvent(req.body, sig, webhookSecret);
} catch (err) {
console.error(Webhook signature verification failed: ${err.message});
return res.status(400).send(Webhook Error: ${err.message});
}
// Process verified event
switch (event.type) {
case 'payment_intent.succeeded':
handlePaymentSuccess(event.data.object);
break;
case 'customer.subscription.deleted':
handleSubscriptionCancelled(event.data.object);
break;
default:
console.logUnhandled event type: ${event.type});
}
res.json({ received: true });
});
Note: express.raw() not express.json() for the body parser. Stripe's signature verification requires the raw body bytes. Using express.json() will parse it first and break verification. These trips people up constantly.
Also worth noting: acknowledge the webhook with a 200 immediately, then process asynchronously. Don't do heavy processing synchronously in the webhook handler, if it times out, the provider will retry, and you'll end up processing events multiple times.
Rate limiting strategy: work with it, not against it
Most APIs have rate limits. The right way to handle them is not just to catch 429s reactively, but to build a request queue that proactively respects the limits.
class RateLimitedQueue {
constructor(requestsPerSecond) {
this.queue = [];
this.interval = 1000 / requestsPerSecond;
this.lastRequestTime = 0;
}
async add(fn) {
return new Promise((resolve, reject) => {
this.queue.push({ fn, resolve, reject });
this.process();
});
}
async process() {
if (this.processing || this.queue.length === 0) return;
this.processing = true;
const now = Date.now();
const timeSinceLast = now - this.lastRequestTime;
const waitTime = Math.max(0, this.interval - timeSinceLast);
await new Promise(r => setTimeout(r, waitTime));
const { fn, resolve, reject } = this.queue.shift();
this.lastRequestTime = Date.now();
this.processing = false;
try {
resolve(await fn());
} catch (err) {
reject(err);
}
if (this.queue.length > 0) this.process();
}
}
// 10 requests per second max
const apiQueue = new RateLimitedQueue(10);
// Usage
const result = await apiQueue.add(() => callExternalAPI('/data', payload));
This is simple but effective for single-process applications. For distributed systems, you'll want a shared rate limit state in Redis, but the same principle applies.
Logging and observability: what you'll need when things break
Good API integrations are observable. You need to be able to answer, after the fact: did this request go out? What did we send? What did we get back? How long did it take?
async function callExternalAPIWithLogging(endpoint, payload) {
const requestId = uuidv4();
const startTime = Date.now();
logger.info('API request initiated', {
requestId,
endpoint,
payloadSize: JSON.stringify(payload).length,
});
try {
const result = await callExternalAPI(endpoint, payload);
logger.info('API request succeeded', {
requestId,
endpoint,
durationMs: Date.now() - startTime,
});
return result;
} catch (err) {
logger.error('API request failed', {
requestId,
endpoint,
durationMs: Date.now() - startTime,
errorType: err.constructor.name,
errorMessage: err.message,
statusCode: err.status,
});
throw err;
}
}
Don't log the full payload in production, it likely contains PII. Log enough to reconstruct what happened (size, request ID, timing, error type) without logging sensitive data.
These patterns have held up across quite a few production integrations. The teams that build robust API-dependent systems tend to share one trait: they think about the failure cases at design time rather than at incident time.
If you're building custom software with complex API integration requirements, payments, communications, ERP connections, third-party data sources, working with a development team that has this kind of thinking baked in matters. Mittal Technologies builds exactly these kinds of integration-heavy systems, and the difference between an integration designed for resilience and one optimized for the happy path tends to show up pretty quickly in production.
