current position:Home>What are the typical application scenarios Dataphin resource management?

What are the typical application scenarios Dataphin resource management?

2022-11-24 14:30:25Ali cloud q&a

What are the typical application scenarios for Dataphin resource governance?




Accept answer 1:

Scenario 1: Storage optimization for long-term unused tables

A company often has tens of thousands of tables. With the change of business and the flow of personnel, some business tables are gradually unused. At the same time, because of storage in multiple partitions, a large amount of storage resources will always be occupied.For example, for the original data of some companies, a partition may have 1T, but the business will only use the latest partition every time, which will lead to a large amount of waste in the storage of historical partitions. You can set the expiration policy of the partitions and offline the historical partitions.operate.

Dataphin resource management function can automatically discover unused data tables or other abnormal data tables, and remind and display them on the management workbench.At the same time, you can view the corresponding data table on the governance workbench, and take measures such as going offline and setting the life cycle.

Scenario 2: Computing and optimizing abnormal computing tasks

Valuable and expensive computing resources that require each task to be used on demand.However, in the actual business development process, there are a large number of abnormal computing tasks, wasting a lot of computing resources, such as brute force scanning (scanning a large number of partitions), data expansion (data output is far greater than input), data skew (some nodes processtoo long), etc.

Dataphin's resource management function monitors the entire link of task execution, automatically discovers these abnormal computing tasks, and reminds and displays them on the management workbench.You can view the corresponding tasks on the management workbench, and take measures such as code optimization and offline for corresponding tasks.

image.png

The above content is excerpted from the "Building Enterprise-level Good Data (Dataphin Intelligent Data Construction and Governance White Paper)" e-book, click https://developer.aliyun.com/ebook/download/7832 to download the full version.


copyright notice
author[Ali cloud q&a],Please bring the original link to reprint, thank you.
https://en.primo.wiki/2022/328/202211241329173845.html

Random recommended