Get the latest perspectives on Microsoft Dynamics ERP software selection process from industry experts.

Discover why COZYROC SSIS+ outperforms Power Automate for bulk ERP data processing, and how both tools work together for seamless automation.

Visit Website View Our Posts

Updated · Originally published

When it comes to automating processes in the Microsoft ecosystem, Power Automate has become a go-to tool for approvals, notifications, and lightweight integrations. But when organizations face the challenge of moving millions of rows of ERP or CRM data quickly and reliably, Power Automate’s row-by-row architecture can become a bottleneck. This is where COZYROC SSIS+, built on SQL Server Integration Services (SSIS), offers a powerful advantage. With in-memory pipelines, bulk load capabilities, and advanced components like Data Flow Task Plus and Table Difference, COZYROC handles large-scale, high-performance data integration that would overwhelm a flow-based tool. The result? Businesses can use Power Automate where it shines, orchestrating workflows and user interactions, while relying on COZYROC SSIS+ to manage the heavy lifting of enterprise-grade data pipelines.

Bulk Data Processing with COZYROC SSIS+.

Example Scenario

You need to load 1M customer records from a flat file into SQL Server, while checking for inserts/updates and applying transformations.

How You’d Do It in SSIS + COZYROC

Create a Data Flow Task

  • Add Flat File Source (additionally COZYROC’s File Transfer Task if pulling from SFTP/cloud).
  • Point it to your file with millions of rows.

Apply Transformations

  • Use built-in transformations (Derived Column, Lookup, Aggregate).
  • Add/Use COZYROC Data Flow Task Plus for implementing metadata-driven pipelines and handling schema drift.
  • Use COZYROC’s Table Difference to automatically detect inserts/updates/deletes without writing complex SQL.

Load into SQL Server

Performance Example

  • SSIS can easily process hundreds of thousands of rows per second because it’s an in-memory pipeline engine.
  • Logging and error rows can be redirected without stopping the pipeline.

End Result: The entire 1M+ record dataset is ingested, transformed, and compared in one execution, fully automated.

See the full video

Power Automate Limitation

How Power Automate Works

  • It is a workflow automation platform (step-by-step trigger → action).
  • Each step processes one item at a time or at best a small page of items.
  • No concept of a bulk data pipeline, instead, you loop through each record (very slow, expensive at scale).

Example with the Same 1M Rows

  • You’d need a For Each loop in Power Automate.
  • Even with pagination, you’re capped (usually 5,000 items per API call).
  • Each record is an API request → execution time and licensing costs skyrocket.
  • Power Automate does not provide streaming memory buffers or transformations like Lookup, Sort, Table Difference.

End Result: Processing 1M rows in Power Automate could take hours or days, and cost significantly more in API/flow run charges.

Key Differences

Feature SSIS + COZYROC Power Automate
Processing model In-memory, streaming ETL pipeline Trigger → Action workflow
Bulk row handling ✅ Millions of rows per execution ❌ Loops through rows (slow)
Transformations ✅ Rich (joins, lookups, diffs, sorting) ⚠️ Limited (basic mapping, expressions)
Parallelism ✅ Yes (Data Flow engine, Parallel Loop Task) ❌ Mostly sequential (some concurrency controls)
Cost efficiency ✅ One package, unlimited rows ❌ Pay per flow run + API calls
Best use case Enterprise ETL, data migration, real-time pipelines Task automation, notifications, small data syncs

Conclusion:

  • COZYROC SSIS+ is built for bulk ETL.
  • Power Automate is best for workflow automation.
  • Use them together when you want PA to trigger/notify around SSIS runs (e.g., via COZYROC Cloud webhooks) but keep the heavy data work in SSIS.

Step-by-Step SSIS Package for Bulk Load CSV to SQL

 

0) Prereqs (one-time)

  • SQL Server with database CozyRocDemo.
  • Visual Studio + SSDT + SSIS.
  • A CSV with 1,000,000 rows:

CSV generator (PowerShell):

$path="C:\data\customers_1M.csv"

"CustomerID,FirstName,LastName,Email,SignupDate,Amount" | Out-File $path -Encoding UTF8

1..1000000 | ForEach-Object {

$id = $_

$fn = "First$($_)"

$ln = "Last$($_)"

$em = "user$($_)@example.com"

$dt = (Get-Date).AddDays(-($_ % 365)).ToString("yyyy-MM-dd")

$amt = [math]::Round((($_ % 10000) / 100.0),2)

"$id,$fn,$ln,$em,$dt,$amt"

} | Out-File $path -Append -Encoding UTF8

Target table:

USE CozyRocDemo;

GO

CREATE TABLE dbo._Customers_1M (

CustomerID   INT            NOT NULL PRIMARY KEY,

FirstName    NVARCHAR(50)   NOT NULL,

LastName     NVARCHAR(50)   NOT NULL,

Email        NVARCHAR(255)  NOT NULL,

SignupDate   DATE           NOT NULL,

Amount       DECIMAL(12,2)  NOT NULL

);

1) Create the SSIS project & connections

  1. New Project → Integration Services Project (e.g., Load1M_Customers).
  2. Create appropriately named SSIS Package. (e.g., Bulk Load - One million records - CSV to SQL)
  3. Create OLE DB connection “SQL Connection”
    • Connection Managers → New OLE DB Connection… → New…
    • Server name: your instance (e.g., .)
    • Database: CozyRocDemo
    • Test Connection → OK
    • Back in the dialog, set Name = SQL Connection → OK

(You can use ADO.NET instead; match the same server & database.)

 

 

 

 

 

 

 

 

 

 

 

 

 

  1. Create Flat File connection “Flat File Connection Manager - CSV”
    • New Flat File Connection…
    • Name: Flat File Connection Manager - CSV
    • File name: C:\data\customers_1M.csv
    • Format: Delimited
    • Column names in the first data row: Checked
    • Header row delimiter: {CR}{LF}
    • Text qualifier: <none>

 

 

 

 

 

 

 

 

 

 

 

 

 

    • Columns tab:
      • Column delimiter: ,
      • Row delimiter: {CR}{LF}

 

 

 

 

 

 

 

 

 

 

 

 

 

    • Advanced tab (datatype/lengths as used in your package):
      • CustomerID → DT_I4 (integer)

 

 

 

 

 

 

 

 

 

 

 

 

 

      • FirstName → DT_WSTR, Length = 50
      • LastName → DT_WSTR, Length = 50

 

 

 

 

 

 

 

 

 

      • Email → DT_WSTR, Length = 255

 

 

 

 

 

 

 

 

 

      • SignupDate → DT_DBDATE

 

 

 

 

 

 

 

 

    •  Amount → DT_NUMERIC, Precision = 12, Scale = 2

 

 

 

 

 

 

 

 

 

2) Build the flow with Data Flow Task Plus

  1. On Control Flow, drag COZYROC Data Flow Task Plus
    • Name: Data Flow Task Plus - Bulk Load csv to SQL
    • (DelayValidation isn’t required here; leave as the default unless you plan runtime metadata changes. See Dynamic Data Flow in Data Flow Task Plus)

 

 

 

 

 

  1. Click the Data Flow tab to open the DFT+ designer and add components:
    • Drag a Flat File Source to the Data Flow canvas → Name: Flat File Source – CSV

 

 

 

 

 

      • Double Click the “Data Flow Task Plus - Bulk Load csv to SQL” to configure it.
      • Connection Manager: Flat File Connection Manager - CSV
      • Default RetainNulls = false;
      • You can click the preview button to see the data’s preview.

 

 

 

 

 

 

 

 

 

 

 

 

      • Outputs:
        • CustomerID (i4)
        • FirstName (wstr, 50)
        • LastName (wstr, 50)
        • Email (wstr, 255)
        • SignupDate (dbDate)
        • Amount (numeric, precision 12, scale 2).
        • Error output is present (default FailComponent behavior if not redirected).

 

 

 

 

 

 

 

 

 

 

 

 

 

    • Drag a COZYROC Database Destination to the Data Flow Canvas → Name: Database Destination - Bulk SQL
      • Connect the path:

Flat File Source - CSV → (output “Flat File Source Output”) → Database Destination - Bulk SQL (input “Database Destination Input”)

 

 

 

 

 

 

 

      • Double click the destination to configure it.
      • Connection: SQL Connection
      • Action: Insert (Action = 0)
      • DestinationTable: "dbo"."_Customers_1M"
      • BatchSize: 10000

 

 

 

 

 

 

 

 

 

 

 

 

 

      • Mappings: 1:1 to external metadata columns
        • Data Flow Task will automatically map like named columns.
        • Destination metadata:
          • CustomerID (i4)
          • FirstName (wstr, 50)
          • LastName (wstr, 50)
          • Email (wstr, 255)
          • SignupDate (dbDate)
          • Amount (numeric, precision 12, scale 2)

 

 

 

 

 

 

 

 

 

 

 

 

 

3) Configure performance & reliability

  • Buffers (Data Flow Task properties):
    • DefaultBufferSize ≈ 104,857,600 (100 MB)
    • DefaultBufferMaxRows start around 50,000–100,000 (tune by row width)
  • Database Destination:
    • BatchSize = 10000; increase if you want higher throughput.
    • Timeout = 0 (no timeout).
  • Minimal logging (optional best practice for larger data loads): use SIMPLE or BULK_LOGGED during the load; revert after if needed.
    • Switch to BULK_LOGGED (or SIMPLE) before loading
      • ALTER DATABASE CozyRocDemo SET RECOVERY BULK_LOGGED;
      • ALTER DATABASE CozyRocDemo SET RECOVERY SIMPLE;
    • After ETL load, switch back to FULL if required
      • ALTER DATABASE CozyRocDemo SET RECOVERY FULL;
  • Indexes/Triggers (optional): disable nonclustered indexes/triggers during the load; rebuild/enable after.
  • Error handling: default error output exists on the source; configure Redirect Row to an error file if you want to retain bad rows without failing the component.

4) Add lightweight metrics (optional)

  • Insert a Row Count transform before the destination to populate RowCountLoaded (package variable).
  • Add an Execute SQL Task after the DFT+ to log into a simple table, e.g. dbo.LoadLog (LoadTime, RowsLoaded, Notes).

5) Run & validate

  • Start Debugging (F5).
  • Confirm green paths; review row counts on Flat File Source - CSV and Database Destination - Bulk SQL.

 

 

 

 

 

 

 

  • Validate in SQL:

USE CozyRocDemo;

SELECT COUNT(*) AS rows_loaded FROM dbo._Customers_1M;

 

 

 

 

See the full video

 

 

6) Optional: scale up and harden

For organizations that depend on Microsoft Dynamics or other ERP platforms, the smartest strategy isn’t choosing between Power Automate and COZYROC, it’s knowing where each excels. Power Automate streamlines approvals, notifications, and user-driven workflows. COZYROC SSIS+ ensures that your ERP data moves in bulk, with speed, accuracy, and minimal cost. By combining the two, you get the best of both worlds: user-friendly automation on the front end and enterprise-grade data integration on the back end. If you’re ready to eliminate the performance trade-offs of large-scale data processing, explore how COZYROC SSIS+ can enhance your Power Automate strategy and take your ERP data pipelines to the next level.

 

Book a Success Call with COZYROC

to learn how your organization can start building faster, more reliable, and more cost-effective ERP integrations today.

 

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.