Defending Image Quality at the Gate
The Silent Degrader
Bad inputs are worse than missing inputs. A missing image is obvious — the pipeline skips it, logs a warning, and moves on. A blank or expired image URL looks valid at first glance: it's a string, it starts with the right prefix, it points somewhere real. But when the AI model receives it as a reference, it generates against a void — producing images that are off-brand, incoherent, or just wrong.
For months, this was a subtle source of quality degradation in our image generation pipeline. Products would occasionally appear as generic shapes instead of their real packaging. Style references would have no effect. And because the pipeline completed successfully, nothing flagged the problem — only the output gave it away.
Where URLs Go Bad
Every image in our system has a lifecycle. At onboarding, we scrape product photos from business websites and try to store them permanently in our own storage. When that succeeds, we have a stable, long-lived URL we control. When it fails — because the image is behind a CDN that blocks automated requests, or the server is slow, or the file is too large — we fall back to storing the original scraped URL.
Scraped URLs are borrowed time. CDNs rotate signed URL tokens. Instagram media links expire. Brand website images get reorganized. A URL that worked during onboarding might return a 404 three days later. When that URL lands in an AI reference slot, the model is effectively flying blind.
Moodboard references have the same problem from the other direction. We pull inspiration images from Instagram and the web, and those sources are even less stable than product sites. An image that anchored our style guidance during the original generation might be gone by the time the user generates their next batch of content.
A Defense in Depth
The fix wasn't a single check — it was a layered defense applied at every point where URLs enter the system.
The first layer is format validation. Before anything else, we now verify that every URL string is actually a valid HTTP or HTTPS URL. This sounds trivial, but relative paths, empty strings, and non-HTTP schemes can all appear when data is assembled from multiple sources. Catching them early prevents them from wasting network budget on HEAD checks or confusing the AI model.
The second layer is reachability verification. For every external URL that hasn't been stored in our own infrastructure, we run a parallel HEAD request — a lightweight probe that checks whether the resource exists without downloading it. We skip this check for URLs we control, since we know they're always available. For everything else, we probe in parallel across all reference slots simultaneously, so the check adds minimal latency to the overall generation.
The third layer is persistence. Before an image ever becomes a reference, we try to copy it into our own storage. A URL we control can never expire. This is the most durable solution, but it doesn't always succeed on the first attempt — and this is where the original system was weakest.
Teaching the System to Persist
The original persistence code made one attempt to download and store each image. If anything went wrong — a momentary network hiccup, a CDN that rate-limits automated requests, a brief Supabase blip — the attempt failed and we fell back to storing the original external URL.
We made three changes to flip the odds in our favor.
First, we added retry logic with progressive backoff. Multiple attempts with short delays between them catch the transient failures that make up the vast majority of real-world network errors. A CDN that returns a 503 under load usually recovers within a second.
Second, we added browser-like request headers. Many CDNs actively block requests that don't look like browser traffic. By including realistic user agent strings and content negotiation headers, we pass the basic bot-detection checks that were silently failing our download attempts.
Third, we made the retry logic apply to storage uploads as well as downloads. A failed Supabase write — even a transient one — used to mean falling back to an external URL. Now we retry the upload independently, so a download that succeeds on the first attempt still has multiple chances to land safely in permanent storage.
Closing the Loop in the UI
Better persistence changes what users see in the dashboard. Before, products with un-persisted images would show whatever the original scraped URL contained — which, if it had expired, triggered an error fallback that rendered as a black tile. This left users confused about which products were ready to use in content generation.
Now, the product dashboard only shows images that are confirmed in our own storage. If a product's images haven't been persisted yet, the slot shows a clear upload prompt instead of a broken image. What you see reflects exactly what the AI will use — no invisible mismatches between the UI and the generation pipeline.
The Broader Lesson
AI models are remarkably good at working with imperfect inputs, which can mask the extent to which bad inputs degrade output quality. When a model receives a blank reference image, it doesn't crash — it improvises. The result looks plausible, which is exactly what makes silent degradation so hard to detect.
The defense-in-depth approach treats URL validation not as a convenience check but as a quality gate. Nothing reaches the model that we haven't verified exists and can be loaded. The pipeline becomes slower to accept inputs and faster to flag problems — which is exactly the right trade-off when the cost of a bad output is a piece of content that represents a real business.
We're building Jiwa AI to make professional content creation accessible to every business in Southeast Asia. If you're thinking about similar challenges in your AI pipeline, we'd love to hear how you're approaching them.