Post reply

Is this If Condition Expression Valid?

  • May 24, 2022 at 2:44 pm

    #4038262

    I hate to be a pain, but I am a beginner at this and am trying to figure out proper syntax for pipelines at this time. I have a If Condition, where I want to copy a table to a data lake if the LSN for that table is larger then the watermark I have recorded. The expression is:

    @greater(activity('lkpGetLSNRange').output.MaxLSN, activity('lkpGetLSNRange').output.TableLSN)

    Theoretically, this should return a True and proceed with the Copy activity I have inside the If Condition's True side. I have not code or objects on the False side of the condition since I want nothing to happen if the condition is not true.

    I am asking this now because the pipeline will not run and while it tells me it will not run, I get no error of any kind logged when it fails, and this If Condition is the last thing I worked on, having other syntax errors I believe I have fixed. The pipeline does publish successfully.

    Let me know if I need to post all of the code for the pipeline for review.

    Thanks,

    Chris

    Learning something new on every visit to SSC. Hoping to pass it on to someone else.

  • May 25, 2022 at 1:05 pm

    #4038994

    I finally found an error message from the pipeline.

    ErrorCode=InvalidTemplate, ErrorMessage=Unable to parse expression 'SchemaName'

    It doesn't tell me enough to find the problem. I tried removing the last item, a Procedure object (to execute a procedure) from the pipeline then ran in debug and got a different error message, but now it would run the pieces of the pipeline and give my messages for each one, if there was a failure.

    I then got:

    Operation on target ForEachTable failed: Activity failed because an inner activity failed; Inner activity name: IfInsertedUpdatedExist, Error: The expression 'greater(activity('lkpGetLSNRange').output.MaxLSN, activity('lkpGetLSNRange').output.TableLSN)' cannot be evaluated because property 'MaxLSN' doesn't exist, available properties are 'firstRow, effectiveIntegrationRuntime, billingReference, durationInQueue'.

    This tell me that what I originally posted about is invalid. I finally figured out the correct syntax:

    @greater(activity('lkpGetLSNRange').output.firstrow.MaxLSN, activity('lkpGetLSNRange').output.firstrow.TableLSN)

    It is quite annoying that the varying syntax is used, like multiple teams are developing pipeline code and not talking to each other, or, if you can believe it, not documenting the reasons for the different syntax and explaining when to use which syntax.

    Now I am up to the latest error message:

    In function 'formatDateTime', the value provided for date time string '1' was not valid. The datetime string must match ISO 8601 format

    The mystery here is that I am not using the function 'formatDateTime' anywhere in this pipeline and I cannot tell where it would be getting a"1" value for input. The object where the error occurs is a CopyData object, and that object merely puts together a schema and table name passed to it, and selects * out of the table to pass to the Sink, which is a file in Azure BLOB storage. What I am doing is selecting records from a list of tables where changes have been made, so I am dynamically providing the table names to the CopyData objects in the pipeline. One of the tables has no DateTime columns in it at all. I can't figure out of the pipeline is assuming date data types for some reason, or what.

    I will continue slogging though, but if anyone has any ideas, I'm all ears.

    Thanks,

    Chris

     

     

     

    Learning something new on every visit to SSC. Hoping to pass it on to someone else.

Viewing 2 posts - 1 through 1 (of 1 total)

You must be logged in to reply to this topic. Login to reply