Import multiple text files into multiple tables

• ariengel

  SSC Veteran

  Points: 247

  More actions

  August 31, 2004 at 1:15 pm

  #60184

  Hi folks,

  Looking for some advice on the best way to do this. I have about 20 text files (of different structures) that need to go into 20 tables. This will run every half hour or so, and only import the files that exist at that time.

  The package I am creating will loop through the directory and find appropriate files. Then it will need to import these files. I can think of three possible ways to do this, but I'm not sure which is best:

  1) Have the 'File Locator' package identify the file that needs to be imported, then kick off the appropriate sub-package that contains the transformation. This would mean there would be 21 packages in all.

  2) Same as above except keep them all in one package. Not sure if this would mean 20 branches to the same workflow or if there is another cleaner way to do this.

  3) Come up with a dynamic tranformation that dynamically changes the destination table and the column mappings. This seems the cleanest but may be the most difficult to build.

  Any suggestions or alternatives?

  Thanks

• philcart

  SSC-Forever

  Points: 47794

  More actions

  September 1, 2004 at 4:50 am

  #521326

  As much as it sounds, keep it simple and go for option 1.

  Then you'll also have the option of manually re-running any of the individual packages.

  If there is any hierarchy to the data then group your packages accordingly.

  --------------------
  Colt 45 - the original point and click interface