Authorization Error
Can you provide guidance on resolving an authorization error that is preventing the refresh of a Power BI dataset within an Azure Synapse Workspace? The error message received is "The Web Activity failed with HttpStatusCode - 'Unauthorized', and…
Not able to connect to on premise sql server using self integration run time in azure data factory
I am able connect to on premise sql server using SSMS tool but while connecting through azure data factory using self hosted integration runtime and linked service getting below error. No mistakes in user name and password .Please help me on this. { …
how to split a csv file based on columns.
Hi Team, There is a scenario in which we have to split columns of a CSV file into unique files in blob. for example let's say I need to create each file based on the combination of (TimeStart + TimeEnd + A) i.e. three columns per file and filename would…
Error using Azure Data Factory to Copy data (Using Upsert) from Azure Blob to Azure SQL database
Hi all, I keep getting this error when I perform an upsert (under copy activity) with this error code: Failure happened on 'Sink' side. 'Type=System.NullReferenceException,Message=Object reference not set to an instance of an…
How can we run the different level of folders in a heirarchy in batches?
I have a doubt if we can approach something like this in ADF. I have created a for each activity where I'm iterating through an array of folders. Let's say I'm passing folder I'd of A and then I'm calling a pipeline in for each that will return sub…
Data Factory - No erorrs
Hi In Data Preview tab I'm getting a message like "Transformation stream is invalid, fix errors to enable data preview. Click to see errors" But when I click to see errors there is no errors. What should I do?
Solutions Architect Expert Certification
Hello, I unfortunately lost access to my microsoft learn account because of my previous company, so I could not renew my AZ104 exam. Today I just passed the AZ-305 exam, I would kindly ask you to provide me the opportunity to renew my AZ-104 exam so I…
Using Entra ID Auth Only in ADF + Azure SSIS IR solution
Hi, We're currently working on complying with Microsoft cloud security benchmark in our tenant and ran into an issue with our Azure Data Factory service. To comply, we need to have only Entra ID Auth on our Azure SQL DBs. We're using Azure Data Factory…
Takes more time for a data to get uploaded
The data that I upload it takes more time to index, that was around only 2kb but it is 1 day now still it is indexing is there anything I should do
pipeline fail during the sink of a copy data to parquet
I have an issue with a pipeline that works just fine with the same resources (storage account, linked server and integration runtime) in a different ADF (TEST env), in PREPROD, the difference is that the IR is shared as it comes from the TEST env, it…
Copy Data Activity failes on datatype uniqueidentifier when Staging is enabled and no column mapping configured.
Using Azure Data Factory Copy Data Activity from Azure SQL Database to Azure Synapse Analytics. The Copy Data Activity failed with InvalidCastException error 'The given value of type String from the data source cannot be converted to type…
ADF deployment issue
Hi, I am using Azure data factory and setup different branches for developers. I have setup adf publish branch as well so that when developers are done with their changes, we can publish the changes to adf publish branch and then from that branch I can…
Azure Data Factory: Fetch xml to return all the attributes not working properyl
Hi, Thanks in advance for your response. I am using copy activity to copy the data from one environment to another environment in dynamics 365. If I dint use any filter in the fetch xml query, it works fine. Otherwise i am getting missing key…
"No result available for stream" error when performing join and previous step uses cached lookup
I have a data flow where I look up and cache rows that have been updated as part of an ETL pipeline. I then want to join that updated data with the existing data in the target database. The problem seems to be that when doing the join operation the part…
Unable to resolve Azure database for mysql name from Data Factory
I am creating a linked service from my Azure Data Factory environment, to access my Azure Database for Mysql database. I select "Azure database for Mysql", and I can see all my mysql servers. I select "From Azure subscription" as the…
Azure Data Factory - Copy data activity failed: Failure happened on 'Source' side
Operation on target Copy data1 failed: Failure happened on 'Source' side. ErrorCode=UserErrorFailedFileOperation,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The file operation is failed, upload file failed at path:…
Salesforce Bulk API Connector
I am using SOQL instead of ObjectApiName. The SOQL is generated using parameter correctly. The older legacy connector works fine for the same pipeline but the new one is giving this error msg: …
i got stuck on deployment is in progress when creating data factory
this is the message that appear to me : status : InternalServerError { "status": "Failed", "error": { "code": "GatewayAuthenticationFailed", "message": "Gateway…
Salesforce V2 Link Service doens't work. error message: a task was canceled Source=mscorlib
I am migrating the ADF salesforce connector to the new V2 connector and now testing it. When testing the connection for the link service, it is good. However, it failed in the copy activity. with message: Failure happened on 'Source' side.…
Creating new linked services returns SqlFailedToConnect
I'd like to start by first stating that this is a continuation of my efforts to connect an Azure SQL DB to Data Factory via the Linked services. Creating new linked services returns InvalidCredentialReference My new error is below in the azure portal in…