November 12, 2019 at 3:25 pm
Good morning everyone,
I'm very familiar with SSIS and it's tools. However, sometimes depending on the project that I'm working on, it seems that my skills are limited on how to make this happens. In addition to phython, powershell and R, what do you audience suggest getting my hands on and expand my skills?
Thanks.
November 12, 2019 at 3:48 pm
That's an excellent question.
My opinion only, but I'd recommend Power Query. From what I've seen coming from Microsoft lately, I wouldn't be surprised if PQ became the defacto language for data wrangling in the future...even in SSIS.
November 28, 2019 at 4:37 pm
Adaptris have a very useful product called Interlok. If you have a situation where you must collect data from numerous clients, Interlok handles this very well. It spins the design ethos on its head so that the clients send the data to either a central file server or a Message Queue. This is useful where a single SQL server may be unable to keep up with the workload. With the file server approach you can then use either Interlok or SSIS to process the files, with the Message Queue approach you have a second Interlok process to handle records from the Message Queue.
It's very useful in this respect where the Hosts of the clients do not want in-bound connections but will allow outbound, tighter security that way. The other boon is that you can set up a Git repository to store the client configurations, that way you can make changes to certain aspects without having to access numerous clients. If the clients themselves have a range of database systems you create a unique configuration for each system, again saving repetition.
November 28, 2019 at 5:16 pm
You could explore the Azure datafactory, flows and databricks
Viewing 4 posts - 1 through 3 (of 3 total)
You must be logged in to reply to this topic. Login to reply