Why an AI-Powered Data Catalog Tool is Critical to Business Success

`
Enterprise Data Catalog
Workshop

Data
is the Critical Foundation
Improve Customer
Engagement
Improve Patient
Outcomes
Reduce Fraud
Reduce
Compliance Risk

© Informatica. Proprietary and Confidential.33
Governance
& Compliance
Because data drives all digital transformation priorities…
Self-Service /
Advanced Analytics
Cloud
Migration
Customer
Experience

© Informatica. Proprietary and Confidential.44
Governance
& Compliance
…And organization rely on data to drive digital transformation
Self-Service /
Advanced Analytics
Cloud
Migration
Customer
Experience
Discovery of critical enterprise
data provided foundation for data
governance program
Global Financial
Services Company
Flipped the 80-20% rule for finding
data to analyzing effort for data
analysts
Enterprise-scale metadata
understanding allows them to
simplify cloud migration projects
Opening up data visibility is fueling
development of new services and
improving quality of patient care
Regional Health
Care Company
Large National
Retailer
Large
Shipping Company

5 © Informatica. Proprietary and Confidential.5
An AI-Powered data catalog is essential to digital transformation
Share your
data knowledge
Understand &
trust your data
Catalog ALL
enterprise data

6 © Informatica. Proprietary and Confidential.6 © Informatica. Proprietary and Confidential.
Data
Steward
How can I manage
metadata for key
enterprise data
assets?
How do I assess
and manage data
quality through the
lifecycle?
Data
Governance Office
How can we
validate and
enforce our data
governance
policies and
definitions?
How can I ensure
data managed
within application
and supporting
processes deliver
value to the
business?
Data
Owner
How can I
discover,
understand and
trust data required
for my analysis?
Data
Consumer
How can IT enable
business discover
data assets with
verified data
quality and
traceability?
Data
Architect
TechnicalBusiness
The questions a data catalog solves

7 © Informatica. Proprietary and Confidential.7 © Informatica. Proprietary and Confidential.7 © Informatica. Proprietary and Confidential.
• Find the data you need with simple,
powerful semantic search
• Understand your enterprise data
with a holistic view
• AI-powered automatic discovery,
classification and business context
• Comprehensive metadata
connectivity for all enterprise data
• Big Data scale and flexible
deployment options
• Open framework for custom
extensions and integrations

Broad
Metadata
Sources
• Technical
• Operational
• Usage
Business
Context
• Glossary
• Policies
• Process
Wisdom
of Crowd
• Comments
• Ratings
• Behavior Knowledge Graph
Business & Crowd
Sourced Curation
AI Curated Catalog
Data Governance
[Data Stewards, Data Architects]
• Associate Business glossary to
technical objects
• Verify business to technical lineage
• Track key data elements compliance
Self Service Analytics
[Data Analysts, Data Scientists]
• Google for enterprise data assets
• Data Lineage, holistic relationship view
• Trust with data profile
• Access to data
Data Asset Management
[Architects, Developers]
• Analyze column-level Lineage &
Change Impact
• View transformation Logic
• Data asset and BI usage
Structure Discovery, Profiling
and Domain Discovery,
Similarity Clustering,
Recommendations
Business Glossary
Associations, Business
Classifications, Annotations,
Comments

Smart Discovery
Semantic Search
Search datasets using business terminology, synonyms,
across related objects and data flows. Example: when
you type “grade”, EDC can suggest “tier” aligning with the
terminology used in the organization.
Data Profiling and Domain Discovery
View data profiling statistics alongside data assets to
understand data quality before using data for analysis.
Profiling statistics include value distributions, patterns,
and data type and data domain inference. Example: you
can search for “tables with emails” to get all tables that
have email information, regardless of the underlying
column names
Facets
Intelligent facets, based on the search results, allow
users to narrow the search to the data sets of interest.
Facets include both system and custom classifications

10 © Informatica. Proprietary and Confidential.
Automatic Data Lineage
…Upstream and Downstream
• Lineage Discovery from:
- Informatica Big Data Management
- Cloudera Navigator
- HW Atlas
- HiveQL
- + Other Enterprise Sources
- Informatica Powercenter
- Microsoft SSIS, Datastage, SAP BO,
Oracle Data Integrator
- BI

Relationship Discovery
360 Relationships View
Get a 360-degree view of data in a knowledge graph that lets you quickly
search, discover, and understand enterprise data and meaningful data
relationships. Automatically discover related data sets, technical,
business, semantic and usage based relationships.
Data Lineage
Interactively trace data origin through business-friendly summarized
lineage views that highlight the end points and all the complex details in
between. A drill-down lineage view expands any lineage path to show
columns and lineage diagram metrics.
Impact Analysis
Perform detailed impact analysis on upstream and downstream data
assets. See impact across data asset, resources and users.

But cataloging and curating
all the enterprise data doesn’t
sound easy!!

Enterprise
Unified
Metadata
Usage Operational
Technical Business

CLAIRE
The Intelligence in the Intelligent Data Platform
Simplify Data
Stewardship
Automate Integration and
data migration
Amplify Analytics
Spotlight Data Risk
Accelerate Governance
Improve Operations
Metadata-driven
Artificial Intelligence

Smart Domains
Uses Column Similarity
Like photo tagging CLAIRE for Columns
Clustering based on column
metadata and Jaccard
Coefficient and Bray Curtis
Similarity
Column similarity based on
data overlap. Large overlap of
distinct values
Similar value frequencies for
overlapping columns

True Type Discovery
…for Entities and Elements
Intelligently recommends other
data sets that are similar to what
they are working on
Discovers data domains
(name, phone, email…) and data
entities (purchase order,
health record…)
Automatically tags data by
learning from users tagging fields
and columns, etc.

…Across Structured, Semi-Structured and
Unstructured
Person(6)Location(3)Date(1) …
Passage from “Chapter 2: The Science of Deduction” from “A Study in Scarlet” by Arthur Conan Doyle
Support for rule based
data domains
Identifying and classifying
entities from both structured
and unstructured data
New unstructured sources
supported: Excel, Word, PDF, text
files, PPT and more formats

Open Metadata APIs
Access metadata knowledge graph with REST APIs

Intelligent Business Term Associations
• CLAIRETM powered automatic
association of business terms
with physical data
• Built on top of auto-data domain
discovery and data similarity
capabilities
• Uses NLP techniques to relate
business terms to field and
column names
• Reduces a tedious manual step in
data governance
Name Asset Type Business Term Recommendation
Mil_ID Column Military ID Number
Med_Nmbr Column Practitioner Medicare number
C_ADJ_KEY Column Claim Adjustment Key
NH_Plan_ID Column National Health Plan Identifier
DSCH_STAT_CD Column Discharge Status Code
NatEmp_ID Column National Employee Identifier

Dataset Certifications
• Data Owners, Data Stewards and
Subject Matter Experts can certify
datasets and data elements
• Includes certification comments
and keyword specification
• Search rankings prioritize certified
data assets
• New search facet for showing
certified datasets only in searches
Certify fit for use datasets
with right business context
to enable high quality
analytics
Data Steward
Use the most relevant and
trusted data for my analysis
Data Consumer

Ratings and Reviews – Yelp for Data
• Catalog users can rate data assets
and write reviews
• User ratings with summaries are
displayed for each dataset
• Reviewer or Data Owner can edit or
remove ratings
• Search across data asset user
reviews
• New search facet for showing
results by ratings
Find the most relevant and
trusted data for my analysis
using reviews provided by my
peers
Data Consumer

Question & Answer – Quora for Data Assets
• Questions can be asked by users
on the data assets
• Answers can be provided by any
user
• Users can mark answers helpful,
which moves the answer up in the
default view
• Question & Answer text is indexed
to help with data asset search
ranking
No more multiple emails and
phone calls for the same
queries on data
Subject Matter Expert
Find experts, ask questions
and get timely answers in the
context of the dataset
Data Consumer

Change Notifications
• Follow data assets of interest, get
notified of changes
• Follow specific changes to objects
- source changes, enrichments,
collaboration updates
• Watch for changes at individual
asset level or for an entire resource
• Change notifications sent via in
app notification center, event email,
periodic digest email
• New tab at Resource level for a
summary of all changes
Get notified when new
assets are introduced to
apply data standards and
associate business
context
Data Steward
Manage change impact
by tracking changes at
the data source level
DB Admin
Stay on top of changes
to important datasets
and reports
Data Consumer

• Duration: 20 minutes
• In this lesson, you will learn the basics of the data catalog. You will learn to search for relevant
data assets using search and dynamic faceting capabilities and explore data assets.
• In this data catalog basics lesson, you will accomplish the following tasks:
• Use the search feature provided by Enterprise Data Catalog to search for an asset and understand the asset details.
• Verify the profiling information displayed for the asset in the Asset Details tab to ensure the quality of data, understand data domains
and similar columns
• Understand the flow of data using Lineage and Impact
• Understand the relationships of the asset
• Read reviews, ratings, or comments on the asset and ask question about the asset.
Lesson#1 The Basics of Enterprise Data Catalog

Lesson #2
Data Catalog for
Self-Service Analytics

Duration: 20 minutes
As a data analyst, you need analyze the customer order report to understand the elements that make up the report as there
are concerns with the report.
In this lesson you will learn how to:
• Search for the customer order report
• Understand how the report was created
• Review and provide feedback on the customer order details table based on different aspects of the asset
• Ask question about the customer order report
• Follow an asset to be informed about the changes that are made to the asset
• Additionally (Optionally)
• In this lesson, you will learn how the Catalog automatically classifies data based on known domains. You will also learn how you can annotate datasets
to further classify data assets along multiple dimensions.
• Search Classified Columns
• Data Domain Overview
• Annotate Data Assets
Lesson #2 Data Catalog for Self-Service Analytics

Lesson #3
Data Catalog for
Data Asset Management

Lesson #3: Data Asset Management
In this lesson, you will learn how to use the new drill down lineage views in the
catalog to visualize data provenance. You will also learn how to use the detailed
impact analysis reports in the catalog to understand impact due to change in data
assets or ETL flows.
Objectives
• Understand Drill Down Lineage Views in the Catalog
• Perform Impact Analysis on Data Assets
• Understand Data Domain Curation

Lesson #4
Data Catalog for
Data Governance

Lesson #4: Data Catalog for Data Governance (optional)
In this lesson, you will also learn how to associate business terminology with
technical assets. Finally, you will use EDC’s discovery features to search for
technical assets using business vocabulary.
Objectives
• Associate Business Terms with Technical Metadata
• Explore Business Metadata in the Catalog
• Search technical assets using business glossary

Why an AI-Powered Data Catalog Tool is Critical to Business Success

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Why an AI-Powered Data Catalog Tool is Critical to Business Success

Similar to Why an AI-Powered Data Catalog Tool is Critical to Business Success (20)

Recently uploaded

Recently uploaded (20)

Why an AI-Powered Data Catalog Tool is Critical to Business Success