1) Connect a Flat File Source to OLE DB Source first.  Connecting automatically will pre-fill some settings.

2) To speed processing between a Flat File and an OLE DB source, in the OLE DB Destination Editor, set the Data Access Mode to Table or view - fast load.

 

 

 

 

3) Methods to remove duplicates in the Flat File source

  • Add a sort and check remove duplicates check box.
  • In OLEDB Destination, redirect duplicate errors to OLE DB Destination Error Output (use the Advanced Editor).

 

4) SQL Resources

5) Another way to remove duplicate rows through the use of the Partition By clause: http://www.sqlservercentral.com/articles/T-SQL/70807/