Microsoft, get this legend a proper external microphone!
@drnighttime
11 ай бұрын
Thank you so much!
@bermagot9238
Жыл бұрын
This is brilliant
@valentinloghin4004
Жыл бұрын
Hi Mark thank you for sharing !!The upsert in delta lake not working for me , at every execution all the records from csv source file are inserted . I used the customers.csv file from AdventureWorks.
@ayocs2
9 ай бұрын
i have production server instead of your newFile, is it okay to do this?
@user-xg5ig8uw2h
Жыл бұрын
can this work for delta table with dynamic schema?
@nadeemrajabali3166
Жыл бұрын
Hi Mark, great tutorial. I want to create hashing on columns except one. That one column is my DataSourceFilename I want to leave it off so if any duplicate data came through other file my hash will capture it. How do I hash leaving one column? my column list is big therefore don't want to select columns individually.
@MSDataFactory
Жыл бұрын
You will have to add a new branch in your data flow and then in the new branch, first add a Select transformation. Select all columns except the column you wish to hash. Then you can add a Derived Column next that will has all of the columns in this branch, which is excluding that 1 column.
@nadeemrajabali3166
Жыл бұрын
@@MSDataFactory Thanks Mark, this worked for me. I also tried using parameter to capture filename and used this parameter after hashing to add filename into a column. I didn't use prebuilt source option to get filename.
@Olishanta
3 жыл бұрын
Is there a way to get this technique to work when I don't know the schema of the lake file? I'm using a parameterized approach to loading tables, so I'm not able to set the incoming schema as you did in this example.
@MSDataFactory
3 жыл бұрын
Yes. The row fingerprint hashing functions can work on dynamic schemas.
@NeumsFor9
2 жыл бұрын
More importantly, is Jerry a family member?
@nigelnaicker7948
Жыл бұрын
why would i use this with Data Flow, vs an Upsert with a regular copy activity?
@SushilChauhan
28 күн бұрын
because he is sinking after doing the some transformation of the data, not the directly ingesting the data into the sink
@satyanarayanagajjala679
3 жыл бұрын
Can we use this technique to get CDC data from Oracle database
@MSDataFactory
3 жыл бұрын
Today, you cannot do that directly in data flows since there is not currently an Oracle direct data flow connector.
@satyanarayanagajjala679
3 жыл бұрын
@@MSDataFactory thank you, is there any other way to pull CDC data from Oracle database using data factory or any other azure tools
@ShashijeevanMP
3 жыл бұрын
Can't we use SQL Server CDC to do this?
@curious-abc-xyz
2 жыл бұрын
That'll work only if the source is SQL Server.
@eddyjawed
4 ай бұрын
First time I seen how you look. You are handsome should show yourself more :D
@DM-nm9dq
3 жыл бұрын
Does this work for all upserts, or this is just for inserts?
@MSDataFactory
3 жыл бұрын
This demo is only for inserts. For this to work with upserts, you need to add an Alter Row transformation and then select the matching columns in your Sink as key columns.
Пікірлер: 22