large amount of data in Azure SQL > U-SQL

I am trying to find the best way to process in U-SQL a query that produces 450 millions rows. I use DataFactory V2 (pipeline) - USQL for transformation

The problem : it takes an eternity to extract it into a csv file and to inject it back to the Azure SQL DB

I wanted to know if it is possible to query directly the Azure database with U-SQL and after injecting the result directly to the Azure DB? it would be faster instead of generating CSV as an imput and a CSV as an output...

thanks

1 answer

  • answered 2018-02-15 17:03 guyhay_MSFT

    Can you share more about the data, the transformation that you're doing, and lastly the number of AUs you're running with. If this data is sensitive you can email me directly at guyhay at Microsoft.com