CSV and MySQL: Chunked Batch Loader Template

A manual Spojit workflow parses a large CSV file and loads its rows into MySQL in bounded chunks, the batch pattern that keeps your database from being overwhelmed by one huge insert.

What It Builds

This template starts from a Manual trigger so you can paste or pass in a CSV file and click Run. A Connector node on the csv connector parses the file into rows, the array connector splits those rows into fixed-size chunks, and a Loop node walks each chunk and inserts it into MySQL one batch at a time. The result is a controlled, repeatable bulk load where a 50,000-row file becomes a series of small, safe writes instead of a single oversized statement.

The Prompt

Paste this into Miraxa, the intelligent layer across your automation, and it builds the workflow and connects the tools for you:

Build a manual workflow that takes a CSV file passed in at run time. Parse the CSV into rows with the csv connector, split the rows into chunks of 500 with the array connector, then use a Loop node to iterate over the chunks and insert each chunk of rows into the customers table in MySQL one batch at a time. Send me a Send Email summary at the end with the total number of rows loaded.

Connectors Used

  • csv - parses the uploaded file into structured rows with parse (and can filter or dedupe rows before loading). Trigger type: Manual.
  • array - the chunk tool breaks the full row list into bounded batches so each database write stays small.
  • mysql - the insert-rows tool writes each batch into your target table inside the Loop.

Customize It

The obvious knobs to change in the prompt are the chunk size (500 is a safe default; lower it for very wide rows or a busy database, raise it if your database handles larger writes comfortably), the target table name (customers), and the summary recipient. If your file has duplicate keys, add "dedupe rows on the email column" to the prompt so the csv connector cleans the data before the Loop begins.

Tips

  • Keep the MySQL node in Direct mode: insert-rows is a predictable single-tool call, so you avoid AI cost and get deterministic batches.
  • Loop over chunks rather than individual rows. One insert-rows call per chunk is far faster and gentler on the database than one call per row.
  • Test with a small file first (a few hundred rows) so you can confirm the column mapping and chunk size before pointing the workflow at a multi-thousand-row export.

Common Pitfalls

  • CSV header names must line up with your MySQL column names. If they differ, add a Transform node before the Loop to rename or reshape each row.
  • A chunk that is too large can still time out or hit a query size limit. If a batch fails, shrink the chunk size in the prompt or in the array node's chunk setting.
  • Re-running the workflow inserts the rows again. If you need to avoid duplicates on re-run, dedupe the CSV first or switch the MySQL step to update existing rows on a key.

Related Articles

Did this answer your question? Thanks for the feedback There was a problem submitting your feedback. Please try again later.