When One Onboarding Pipeline Becomes Two

A Single Job Doing Two Different Jobs

When we first built Jiwa AI's async onboarding pipeline, the design was deliberately unified. A business owner submits a URL, we kick off a background job, and the job pulls whatever social data it can find — Instagram profile, TikTok videos, website content — and feeds it all into the AI analysis engine. One job type, one database table, one Cloud Run executor.

That simplicity served us well through early iterations. But as TikTok onboarding matured into a real, distinct product path — with its own OAuth flow, its own data shape, and its own error semantics — the seams started showing. Fields that only existed for TikTok jobs sat empty on every Instagram job. Instagram-specific WhatsApp notification logic ran inside code that also handled TikTok. The unified model had become a shared dumping ground.

The Boundary Revealed Itself

The clearest sign that a split was needed wasn't the code getting messy — it was the fact that the two flows had genuinely different operational profiles.

Instagram onboarding is tightly coupled to WhatsApp. When a user connects their Instagram via a WhatsApp link, they expect to receive progress updates as the pipeline runs — a message when scraping starts, another when brand analysis completes, another when the content calendar is ready. That notification rhythm is a core part of the experience. WhatsApp-initiated onboarding can also happen without a logged-in user, relying on phone number as the identity anchor.

TikTok onboarding is entirely web-first. Users connect through the dashboard, they're always authenticated, and there's no real-time notification channel to maintain. The job runs silently, updates its status in the database, and the frontend polls for completion. It's a simpler contract.

These aren't superficial differences. They mean the two job types have different input shapes, different error recovery strategies, and different definitions of success. Keeping them in a single executor meant every change touched concerns it didn't own.

Two Models, Two Executors, One Shared Engine

The split followed a clean three-layer decomposition.

At the data layer, we added a dedicated database model for TikTok onboarding jobs. The existing model — now unambiguously an Instagram-and-website onboarding job — shed its unused TikTok fields. Each model carries exactly the fields relevant to its platform: the Instagram model retains WhatsApp phone number, language, and notification preferences; the TikTok model carries only the session-scoped credentials and source context it needs.

At the executor layer, a new job runner handles TikTok jobs exclusively. It shares no code with the Instagram runner beyond calling the same underlying AI orchestration engine. No notification callbacks, no WhatsApp messaging, no phone-number-based identity resolution. The executor is a clean, narrow wrapper around the shared AI pipeline.

At the trigger layer, the infrastructure that routes jobs to Cloud Run workers gained a third entry. Instagram and website onboarding continue to route to the existing worker. TikTok jobs route to a new worker configured independently — separate scaling knobs, separate environment variables, separate observability.

Why Parallel, Not Sequential

One concrete improvement that fell out of the split: the job status polling endpoint, which the dashboard calls repeatedly while a job runs, previously checked each job type one at a time. Instagram first, then TikTok, then reels — three sequential database round trips on every poll.

With the models now clearly separated and the access pattern explicit, we restructured the endpoint to issue all three queries simultaneously. For a user actively watching an onboarding job progress, this cuts the response time to the slowest of three fast queries rather than the sum of all three. Small improvement in isolation, but multiplied across every poll cycle it meaningfully reduces the perceived latency of the progress indicator.

Shared Logic Stays Shared

The most important thing we didn't do was duplicate the AI engine itself. The brand analysis, influencer matching, content calendar generation, and image pipeline are exactly the same for both platforms. Both job runners call the same orchestration function, passing different input parameters depending on what social data they have.

This matters because the value of the split is in separating operational concerns — not in fragmenting the AI intelligence. We want Instagram and TikTok businesses to receive equally good AI analysis and equally well-matched content. Keeping the AI layer unified ensures improvements to the core engine benefit both paths automatically.

Cleaner Services, Clearer Ownership

The result is two onboarding services that each know exactly what they are. The Instagram service owns the WhatsApp notification contract. The TikTok service owns the web-first, silent-execution contract. Neither knows about the other.

When something breaks in TikTok onboarding, we look in one place. When we want to add a new WhatsApp notification stage to the Instagram flow, we don't have to worry about accidentally changing behavior for TikTok users.

That kind of confidence — the ability to change one service without fear of breaking another — is what good service boundaries actually buy you. The lines weren't drawn arbitrarily. They were drawn where the product differences were already drawing them.