An Azure service for ingesting, preparing, and transforming data at scale.
Can a single ADF pipeline trigger multiple Azure ML pipelines in parallel when their inputs and outputs are independent?
I have an Azure Data Factory pipeline that orchestrates a multi-stage classification workflow. Right now every activity is chained one after another with success dependencies, so the whole pipeline runs strictly sequentially and takes a long time…
Azure Data Factory
Data factory - Azure Key Vault (Preview) Activity - bug?
The Azure Key Vault Activity in data factory (in preview) does not do anything - settings tab just says azure-key-vault-activity-settings works! Is this expected?
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
How to copy data from google secops platform using ADF
Hi there, How to copy data from google secops platform using ADF Thanks & Best regards, zmsoft
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Any changes happened in the backend of Parse JSON Mapping data flow in ADF in Feb Week 2
Hi All, Is anyone else faced an issue with Json array not being parsed correctly in the ADF Mapping data flows. Earlier the string worked fined and able to parse the columns but it now shows Null Json Array is like [{"abc":"def",…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Some of the CDS connection copy activities in the Azure Data Factory pipelines have been stuck in the queue for a long time.
Today, the copy activities using the CDS connection have been stuck in the queue for a long time. Previously, these activities used to complete within minutes, but today they have been in the queue state for 3–4 hours.
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Azure Data Factory: Inquiry on New Undocumented Property 'trustModeClaimForMi'
It looks like PG added a new undocumented property on data factory resource properties: "trustModeClaimForMi": { "value": "Enabled", "editable": true } Any ideas about this and any ETA when…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Storage firewall blocking Snowflake stage access for ADF Oracle to Snowflake pipeline
Hi Azure Team, Our ADF pipeline (Oracle to Snowflake) is failing during Snowflake load with stage access error. Observed error: Operation on target Oracle to Snowflake_2024_v1 failed with SnowflakeOperationError. Failure using stage area. Status Code:…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
ADF Pipeline issues [Sev 2 ] [MAQ Software] - [ADF Pipeline Execution Delay – Pipelines Persistently Queued During Refresh Window]
Issue Description: We are currently facing an issue where multiple ADF pipelines are remaining in the Queued state for an extended duration during 5/29 refresh process. Pipelines that normally complete within a few minutes are now queued for more than 2…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Data from FoxPro Onprem server to ADLS
Hi Team, We are using FoxPro 9.0 x3 server in our organization that has some logs with emaint data. I would like to know if there is any way I can download or import the data (.dbf files) to ADLS in Azure using the ADF. Fyi, I have server name and…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Azure Data Factory PostgreSqlV2 connector incorrectly serializing DATE columns as internal .NET type name
We are experiencing a sudden issue with an Azure Data Factory pipeline that extracts data from a PostgreSQL source and writes to Parquet files. Since yesterday, the pipeline has started failing without any deployment or configuration changes on our…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Join: Microsoft Azure Q&A Champions Program
The Microsoft Azure Q&A Champions Program recognizes and empowers a global community of internal and external Azure experts who help customers succeed by providing high-quality, trusted answers on Microsoft Q&A. The program scales expert led…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
AI answer
Quote character and escape character in ADF
Hi, We need to load data from delimited file in storage account to table in dedicated sql pool.. All the record values are enclosed with double quotes (" " ) and column delimiter is pipe (|) For example :- …
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Autoupdate on SHIR seems to have re-nabled and forced a restart of the SHIR service
On April 27th we had an issue with batch jobs that caused delays on key business process, which drew quite a bit of attention. After some initial investigation it was found that the overnight jobs failed because the SHIR they were running on had the SHIR…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
I want to document ADF pipelines runs / triggers
I want to document ADF pipelines runs / triggers in excel or PowerBI (for report purpose) and note what the data is being used for and source and target data as well as trigger times.. how often and when does it run. Is there any way that I can get these…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
ADF when publishing, the arm templates generation failed with Corporate Public network Disabled errror- how to resolve?
ADF settings show networking with Public Endpoint ADF is using Managed Virtual networks with private endpoints to reach out of other AZ resources When publishing the arm templates generation failed with this error: Generating ARM templates …
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
FoxPro 9.0 to Azure Storage via Azure Data Factory
Hi Team, We are using FoxPro 9.0 x3 server in our organization that has some logs with emaint data. I am unable to find the database details. I would like to know if there is any way I can download or import the data to SQL server in Azure or Azure Data…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Best Approach to Move Selected Azure Resources Between Dev and Prod Subscriptions
Hi Everyone, I currently have a Dev Azure subscription containing the following resources: Azure Data Factory with 100 pipelines Azure SQL Database with: 150 tables 100 stored procedures 20 views 10 functions 10 Logic Apps ADLS Gen2 storage account Azure…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
How to fix "Status code 400, "{"error":{"code":"InvalidSubscriptionId","message":"The provided subscription identifier 'resourceGroups' is malformed or invalid."}}""
While testing connectivity in Azure Data Factory (ADF), the dataset test connection succeeds, but the data flow test connection fails with the following error: Status code: 400 { "error": { "code":…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Azure Data Factory UserErrorSourceQueryTimeout caused by Linked Service Azure Database for Postgresql
Hello, Recently, I have encountered the following error for Lookup or Script activities when the activity takes longer than 15 minutes to complete: Operation on target Lookup1 failed: Failure happened on 'Source' side.…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
AzureDataLakeStore does not support MSI authentication in data flow.
We are using MSI in Azure data but when we make a change to this pipeline even if not related (triggers) we are always seeing the same issue: AzureDataLakeStore does not support MSI authentication in data flow. Would it be possible to make a change to…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.