Codewerk.
Get a quote
Home/Blog/Importing 100,000 products without killing the shop

Importing 100,000 products without killing the shop

One request per product is a 28-hour import. Bulk payloads and the sync API turn it into twenty minutes — if you get the batching right.

Photo: free stock photography (Unsplash licence) — see imprint

Stop making one request per row

Every HTTP request carries auth, routing and transaction overhead. At 100,000 products that overhead is the entire runtime. The sync endpoint accepts many operations in one payload — that is the whole trick.

Find your batch size empirically

Batches of 50–200 usually win. Too small and you are back to overhead; too large and you hit memory limits and lock contention, and one bad row rolls back the whole batch. Measure on staging with production-shaped data.

Upsert, and be idempotent

Send a deterministic ID derived from your source system, so re-running the import updates rather than duplicates. An import you cannot safely re-run is an import you will be afraid to fix.

Indexers are the hidden cost

After the write, Shopware reindexes. On a big import, let the indexers run once at the end rather than after every batch, and schedule the whole thing for a quiet hour. Otherwise your import is also an outage.

Key takeaways
  • Sync API + batches of 50–200, measured on staging.
  • Deterministic IDs make the import re-runnable.
  • Defer reindexing to the end of the import.

We do this for a living — Shopware, Node.js, React, ERP integration and automation for B2B.

Talk to an engineer

// Keep reading

Related articles