Basic Setup (manual splitting)
Steps:1
Import your file
- Use the Pull from Excel file or Pull from CSV file step to upload your dataset.
2
Assign batch numbers
- Add an Add row numbers step.
- Set it to repeat each number 20,000 times.
- This will tag rows 1–20,000 with 1, 20,001–40,000 with 2, and so on.
3
Filter and export each batch manually
- Add a Filter rows step for each batch (e.g., where “Row Number” = 1).
- Follow it with a Generate CSV file step to export that batch.
- Repeat for each batch needed.
Advanced Setup (automated batch export)
To automate this over time, especially for files received regularly: Steps:1
Use a Parabola table to store the last processed batch number.
2
In each run:
- Pull the large file and assign batch numbers as above.
- Use a Find overlap step to match rows with the next batch number.
- Export the batch using a Generate CSV file step.
- Update the batch tracker in the Parabola table to increment the number.
3
Repeat runs until all batches are processed.