Post reply

Massive import is bogging down. Why?

  • April 14, 2009 at 9:02 am

    #977308

    Yeah, I already have the recovery mode set to "Simple" so there shouldn't be any logging. I'm thinking that it is probably the primary key index on the table. Would that cause it to slow down as the table grows with check constraints being checked? I'm not sure what is going on in the background.

  • April 14, 2009 at 9:39 am

    #977347

    Could be the PK, but is it the only key on that table

  • April 14, 2009 at 9:49 am

    #977358

    Yeah, only key and the only index

  • April 14, 2009 at 10:41 am

    #977421

    Vishal, you're the man!

    I stopped my process (was scared to death since in 20 hours it had loaded 60 million rows) and changed my max commit size and the Rows per batch in the oledb destination to 100000 each. Then removed the check constraints and removed the primary key index on the table and made sure the table lock was checked and in 5 minutes I had surpassed 60 million rows!

    Many Thanks!

  • April 14, 2009 at 11:27 am

    #977465

    No problemo.

  • April 14, 2009 at 11:33 am

    #977473

    I was actually able to complete the process of 110 million records in 17 minutes

  • April 14, 2009 at 11:37 am

    #977477

    Cool. Now i also have numbers to back my technology suggestions for using SSIS for transfers.

Viewing 7 posts - 16 through 21 (of 21 total)

You must be logged in to reply to this topic. Login to reply