Manage data tiers and retention in Microsoft Sentinel

Data you collect into Microsoft Sentinel (SIEM) and Microsoft Defender XDR is stored in tables. The Microsoft Defender portal lets you manage the retention period and the storage costs associated with your data. You can manage retention and costs when you:

• Configure data connectors to send data to Microsoft Sentinel or Microsoft Defender XDR.
• Manage your existing tables and data.

This article explains how to manage table retention and tier options in the Microsoft Defender portal to optimize security operations and reduce costs in Microsoft Sentinel and Microsoft Defender XDR.

Which tables can you manage in the Microsoft Defender portal?

This section describes the table types you can manage in the Microsoft Defender portal.

How data tiers and retention work

You can retain data in Microsoft Sentinel in one of two tiers:

• Analytics tier: This tier makes data available for alerting, hunting, workbooks, and all Microsoft Sentinel features. It retains data in two states:

  • Analytics retention: In this "hot" state, data is fully available for real-time analytics, including high-performance queries, analytics rules, and threat hunting. By default, Microsoft Sentinel and Microsoft Defender XDR retain data in this tier for 30 days. You can extend the retention period of all tables to up to two years at a prorated monthly long-term retention charge. You can extend the retention period of Microsoft Sentinel solution tables to 90 days for free.
  • Total retention: By default, all data in the analytics tier is mirrored to the data lake for the same retention period. You can extend the retention of your data in the lake beyond the analytics retention, for up to 12 years of total retention at a low cost.

• Data Lake tier: In this low-cost "cold" tier, Microsoft Sentinel retains your data in the data lake only. Data in the Data Lake tier isn't available for real-time analytics features and threat hunting. However, you can access data in the lake whenever you need it through KQL jobs, analyze trends over time by running scheduled KQL or Spark jobs, and aggregate insights from incoming data at a regular cadence by using summary rules.

• XDR data: By default, Microsoft Defender XDR threat hunting data is always available in the Analytics tier for 30 days. You can extend retention of this data in the Analytics tier up to 90 days, which incurs ingestion costs, while storage remains free. Extending retention beyond 90 days in the Analytics tier also incurs storage costs. You can also ingest exclusively into the Data Lake tier, but the data remains available in the Analytics tier for 30 days. Ingesting XDR data directly into the Data Lake tier is more cost-effective, but it involves ingestion, storage, and processing costs. In this option, customers still get 30 days' worth of data in the Analytics tier at no additional cost.

For more information about the differences between these two retention types, see Compare the analytics and data lake tiers.

This diagram shows the retention components of the analytics, data lake, and XDR default tiers, and which table types apply to each tier:

For more information about the Microsoft Sentinel data lake, see What is Microsoft Sentinel data lake.

Use Table insights for table-level observability

In the Microsoft Defender portal, Table insights is a built-in monitoring view on the Tables page that helps you understand ingestion health at the table level for the selected Microsoft Sentinel workspace. It gives you quick observability signals so you can identify ingestion and cost patterns before you change tier or retention settings.

Table insights includes signals such as:

• Ingestion volume by tier
• Day-over-week ingestion fluctuations
• Top tables by daily ingestion volume
• Last data received
• Estimated daily ingestion cost
• Volume anomaly

Use these signals to detect connector failures, investigate cost spikes, validate new connector onboarding, and identify candidates for right-sizing retention and tier.

For setup, access, and interpretation guidance, see Configure table settings in Microsoft Sentinel.

Compare the analytics and data lake tiers

This table compares the Analytics and Data Lake tiers and their key characteristics:

What happens when you modify table settings

You can switch a table's tier and retention settings at any time.

When you change a table's tier from Analytics to Data Lake, all real-time analytics and hunting queries stop working.

When you shorten a table's total retention, Microsoft waits 30 days before removing the data, so you can revert the change and avoid data loss if you made a configuration error.

When you increase total retention, the new retention period applies to all data that was already ingested into the table and wasn't yet removed.

When you change the analytics retention settings of a table with existing data, the change takes effect immediately.

Example:

• You have a table in the Analytics tier with 180 days of analytics retention. By default, total retention is also set to 180 days.
• You change analytics retention to 90 days without changing total retention, which remains 180 days.
• Microsoft Sentinel automatically removes the last 90 days of data from analytics retention, but continues to store data that's 90-180 days old in the Data Lake tier.

Manage XDR data in Microsoft Sentinel

By default, Microsoft Defender XDR retains threat hunting data in the XDR default tier for 30 days. This data isn't ingested into the Analytics or Data Lake tiers by default. If you extend the retention period of supported XDR tables beyond 30 days and up to 90 days, Sentinel ingestion costs apply, but there are no additional storage costs. The tables are created in your Microsoft Sentinel workspace in the Analytics tier and mirrored to the Data Lake tier.

If you enable the Microsoft Sentinel XDR connector in the Azure portal, the tables you select during setup are automatically ingested into the Analytics tier and mirrored to the Data Lake tier. The default retention is 30 days, and you can extend it up to 12 years. For a list of tables, see Microsoft Defender XDR integration with Microsoft Sentinel. You can ingest supported XDR tables that you didn't select during connector deployment into the Analytics tier and mirror them to the Data Lake tier by setting retention to more than 30 days.

If you don't enable the Microsoft Sentinel XDR connector, XDR tables aren't automatically ingested, but you can still ingest them by setting Analytics or Data Lake tier retention to more than 30 days in the Microsoft Defender portal.

You can choose to ingest supported XDR tables exclusively into the Data Lake tier by selecting the Data Lake tier option when configuring retention settings. For more information, see Configure data retention and tiering.

Stop ingesting data into the Analytics tier by resetting analytics retention and total retention to the default 30 days. This action disables the connector in the Azure portal.

For more information about managing your tables and data, see Manage your existing tables and data.

XDR data retention and costs

The following tables summarize the free retention periods and cost implications for the different tiers in Microsoft Sentinel:

For more information about billing and costs, see Understand the full billing model for Microsoft Sentinel.

In the following examples, XDR data is available through advanced hunting for at least 30 days, regardless of the retention settings in the analytics or data lake tiers.

Next steps

Learn more about:

• Configuring log table tiers and retention settings
• Creating a search job to retrieve data matching particular criteria