June 3, 2014 at 12:00 pm
I would like to fetch the data flow component name while package is executing. Since system variable named [System::SourceName] only fetches name of the control flow tasks? Is there a way to capture them?
Thanks
June 3, 2014 at 1:15 pm
You can only get the name of a task.
Are you trying to determine which component gave an error?
Need an answer? No, you need a question
My blog at https://sqlkover.com.
MCSE Business Intelligence - Microsoft Data Platform MVP
June 3, 2014 at 2:19 pm
yes, I would like to log all the components in the data flow task. Is it possible to log them?
June 3, 2014 at 2:22 pm
Confusing Queries (6/3/2014)
yes, I would like to log all the components in the data flow task. Is it possible to log them?
What exactly do you want to log?
Need an answer? No, you need a question
My blog at https://sqlkover.com.
MCSE Business Intelligence - Microsoft Data Platform MVP
June 3, 2014 at 5:53 pm
Koen Verbeeck (6/3/2014)
Confusing Queries (6/3/2014)
yes, I would like to log all the components in the data flow task. Is it possible to log them?What exactly do you want to log?
I would like to log details like when each component execution started and finished, log any error in case package fails. Basically, I would like to implement preExecute, postExecute and onError events on Data flow components.
June 3, 2014 at 6:13 pm
The data flow doesn't work like a query, or like typical programming logic.
Don't think 'steps', think 'places in a stream'.
The data flow streams data from the source, through all the components, and finally to the outbound. So while the second item in the stream may start 500 ms after the first one, the first one won't have completed yet. The only time this logic breaks down is when you do aggregations or sorts mid-stream, where it has to gather the entire stream up to be able to process the results accurately... basically something you want to avoid completely unless you're working against a couple of flat files and you're delivering to another flat file.
Getting the start/end time of each component because of this is not effective. It's not going to show you what's eating the time in the process, because basically they're all active simultaneously and steps upstream may be stuck waiting on a downstream one. I'm assuming that's why you'd want to do something like this, there's really not a lot of other reasons.
EDIT: Also, errors are logged if the package fails. Turn on the logfile in the job step if you want to have easy access to the results of a package run. If you're redirecting failures deliver those to a table that can handle the inbound metadata (or a giant VARCHAR(MAX) field set if it's on the source pickup). You shouldn't be creating any error events to simply see the error. You would only do this if you expect the package to be able to fix errors programatically... something I've found to be overly difficult compared to being notified (via the job failing) and eyeballing the problem (seeing the step log to determine the nature of it, or the redirected rows).
Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.
For better assistance in answering your questions[/url] | Forum Netiquette
For index/tuning help, follow these directions.[/url] |Tally Tables[/url]
Twitter: @AnyWayDBA
Viewing 6 posts - 1 through 5 (of 5 total)
You must be logged in to reply to this topic. Login to reply