尊敬的 微信汇率:1円 ≈ 0.046078 元 支付宝汇率:1円 ≈ 0.046168元 [退出登录]
SlideShare a Scribd company logo
© 2018 iED
1
INTEGRATE. ENABLE. DRIVE.
© 2018 iED
2
Agenda
Industry Challenges
Reusable Assets
1
2
4
5
6
Solution Architecture
7
8
3
Platform Supported
DevOps
Value Proposition
iED
© 2018 iED
3
Strategize, Assess & Plan Design, Build & Validate Run & Optimize
CLOUD
OPERATIONS
SERVICES
DATA LAKE
CONSULTING
SERVICES
DATA
MIGRATION, DATA
LAKE DESIGN &
BUILD
iED | Your Trusted Technology Partner
• Cloud strategy
• Fit Gap Assessment
• Migration Strategy
• Data Lake & Analytical
Roadmap
• Data Lake Design & Build on Cloud
• Data Migration, Transformation, &
Validation
• Data Security Implementation
• Data Management & Monitoring
• Service Management
• Security and Identity Management
• Capacity Planning and Optimization
• Performance and Troubleshooting
iED was founded after several decades of the learning by industry leaders to create production grade data lakes that accelerate
advanced analytics and machine learning to play a critical role in transforming enterprises.
© 2018 iED
4
Industry Challenges & Highlights
Data certification
Inadequate “CloudFirst”
focus
Rapidly evolving technology
choices
Bringing agility and scalability
Fit-gap analysis leading to
technology selection
Rule driven data quality engine
• Pre- orchestrated ingestion, storage and consumption engine
• Automated monitoring and scheduling
• Configurable, functional, and business rules
• Functional and technical validation
• Layer based balancing
• Responsive data architecture
• Intelligent data management
• Delivery at scale
Current Challenges Our Solution Key Highlights
• Searchable meta data and lineage
• Enterprise semantic layer
• Domains: Multiple
Self service *
Time to value
Pre-built domain taxonomy
Automation and asset based
implementation
• Integration Framework
• Audit, Balancing & Control
• Data lineage *
• Continuous integration/deployment
* Future road map
© 2018 iED
About VISVA
 Quickly and easily set up massive ingestion with an
intuitive GUI
 Metadata driven data ingestion
 Maximize throughput using parallel threading
 Web-based control features including restart-ability,
throughput monitoring and notifications
Ingestion Validation
Transformation Consumption
 MD5 Checksum based validations
 Multi-level data validation report (row-count,
row-level, cell level)
 Optimized data comparator using custom code
(spark, native)
 Metadata validation
 Data Profiling and Standardization
 Dynamic creation of Logical views
 Point In Time view of source data
 Support for multiple file formats – AVRO, ORC, etc.
 Analytical data sets
 Augmented Data Discovery
 Enterprise semantic layer
 Data Visualization and self service
5
“VISVA is iED’s Cloud Data platform that facilitates modernization of enterprise Data Assets to maximize performance
and reduce cost. VISVA frameworks brings 4x performance improvement and more than 60% effort savings.”
© 2018 iED
VISVA | Implementation
6
Platform supported
Compute Storage Sources
HDInsight
Hadoop
Data Lake Store Mainframe
Databricks SQL Database File Systems
Data Lake
Analytics
SQL Data
Warehouse
RDBMS
Native Spark Blob Cloud
Azure Batch Analysis Services Web
Hadoop
Salesforce
Packages
Basic (option
to select one of
the four below)
Advanced Premium
Ingestion
  
Transformation
 
Validation
 
Consumption

70%
Automated
Data
Ingestion
50%
Faster Time
to
Deployment
60%
Automated
Schema
Generation
99%
Data
Accuracy
© 2018 iED
7
Solution Architecture
© 2018 iED
8
Data
Accquisiton
Perform Data
Standardization to create
staging layer
Landing Layer
Authentication,
Authorization,
Accounting & Data
Protection
Data Processing Layer
AES-256
Consumption Layer
Source Layer
Appliance
RDBMS
Flat Files
Audit Balance and
Control Framework
Monitoring and
Administration
Active Copy Layer
Staging Layer
Batch
Data
Consumption
Azure Data Lake Store
Azure Active
Directory
Load
Curated
data to
ADW using
Polybase
Azure Monitor
Express
Route
AES-256 AES-256
Azure Data Lake Store
Azure Data Lake Store Azure Data Lake Store
AES-256
Curation Layer
Create point in
time
view of data in
active copy
layer
Apply ETL transformation
to created curated layer
Web
Real
Time
Express
Route
ADF V2
Integration
Framework
1
2
3
4
5
6
7
Data Comparator
Tool
Azure SQL DB
Azure SQL Data
Warehouse
8
VISVA | Solution Architecture
Excel BI
Power BI
SSRS
Data Zen
SharePoint BI
Visualization
© 2018 iED
9
VISVA | Architecture Highlights
• Data Integration Framework for extracting and loading large volume of data from a
variety of data sources onto Azure/AWS
• Dynamic Metadata Synchronization between On-premise and Azure/AWS
• Audit, Balance and Control - framework is capable of providing detailed Audit Reports
including process ran, success rates and failures along with functional/business rule
validation
• Monitoring & Scheduling - The Monitoring Workbench provides the facility to
automate the workflows involved in the enterprise operational process through its
scheduler
© 2018 iED
10
Reusable Assets
© 2018 iED
11
Azure WebApp -
WebJob
1
CDI Framework
Configuration
Database
Azure WebJob to execute the
Integration Framework
Executable
Integration Framework reads the
Configuration Database for Data movement
and Orchestration configurations
2
3
Integration Framework generates the
ADF V2 pipelines for Data Movement
and Orchestration activities
Data Movement
ADF V2 Pipelines
Workflow orchestration
activity
Transform Data in Active Copy
Zone using Databricks activity
Datahub Landing
Zone
ADLS
On-Premise
Sources
Netezza, DB2, SQL
Server, Flat Files,
Web
Datahub Staging
Zone
ADLS
Datahub Active
Copy Zone
ADLS
a
b
c
Provisioning of
Sources in Azure Data
Lake Store
4
Apply ETL
transformation and
build data marts
5
Databricks Spark
Activity
Datahub Curation
Zone
ADLS
Datahub
Consumption
Zone
6
a
b
Polybase
Activity
c
VISVA | Integration Framework
Metadata Driven Integration Framework to Provision a Data Source from On-Premise to Azure/AWS
© 2018 iED
12
Asp .NET
Azure Web App
Custom Scheduler
Azure Automation
Runbook
Runbook polls to check if
schedule has arrived for a
workflow to execute
Custom Job Status Portal
to monitor the status of
workflow
Azure Data Factory V2
ZEA Pipeline
Runbook triggers the ADF
V2 pipelines
Validate the workflow
schedule details from
Schedule DB
Workflow
Schedule Table
Workflow
Status Table
ADF Pipeline
Status Table
ADF Activity
Status Table
Production Support
Team
Support team generates
Reports/metric by querying
operational Audit tables
Customer reviews
Status report
Support team
publish
health report to
customer
1
2
3
4
ADF V2 pipeline executes
workflow and updates
the operational audit
details in Audit Database
Operational Audit Database
7
8
9
Functional Rule
Database
Transformed and prepared data is validated
against functional rules defined in Rule database
Functional Reconciliation
Framework
5
6
Functional Reconciliation
activity is triggered post
workflow execution
VISVA | Audit Balance & Control
Automated Configurable ABC Framework which governs and certifies the data across the ecosystem
© 2018 iED
13
Integration framework reads the
config. database and generates
data lineage pipelines
Data Engineers
ETL workflow captured
in SQL DB template
Integration Framework
configuration database
SQL DB project template in
Visual Studio
ADF V2 pipelines are
deployed in to Data
Factory
VSTS GIT VSTS CI/CD Pipeline
Visual Studio Team Services
Azure Data Factory V2 Pipeline
CI/CD process deploys the
SQL DB project changes to
config. database
1
6 5
4
3
2
Integration Framework
ETL workflow SQL DB
project checked-in to
VSTS GIT
Azure Cosmos Graph DB
ADF V2 pipelines
updates the Graph DB
with lineage details
7
Data Lineage
Data Lineage captured
through custom API or
enterprise lineage tools
VISVA | Data Lineage (Future Roadmap)
© 2018 iED
14
VISVA Demo
© 2018 iED
15
VISVA | UI
© 2018 iED
16
DevOps
© 2018 iED
17
Plan + Track Release
Defect Management
Track Progress
Manage work
Plan work
Project Starts
Automation
functional testing
environment
Automation Integration
testing environment
Pre-production
Environment
Staging
environment
Track defect
Manage defect
Defect matrix
Build
Version control
Unit testing
Write code
Develop + Test
Report status
Defect Management
Track Defects, tasks, and blocking
issues
Project Management
Project plan by creating a backlog of
user stories that represent the work
Release Automation
Automated release pipelines in
which you can reliably test and
release on much shorter cycles.
Development
Develop, unit test and build
automation
VISVA |Continuous Integration/Deployment
© 2018 iED
18
www.ienergydigital.com

More Related Content

Similar to 5. iED Cloud Services.pdf

Single View of Well, Production and Assets
Single View of Well, Production and AssetsSingle View of Well, Production and Assets
Single View of Well, Production and Assets
John Archer
 
Azure SQL Database Managed Instance - technical overview
Azure SQL Database Managed Instance - technical overviewAzure SQL Database Managed Instance - technical overview
Azure SQL Database Managed Instance - technical overview
George Walters
 
Dev show september 8th 2020 power platform - not just a simple toy
Dev show september 8th 2020   power platform - not just a simple toyDev show september 8th 2020   power platform - not just a simple toy
Dev show september 8th 2020 power platform - not just a simple toy
Jens Schrøder
 
Azure Data.pptx
Azure Data.pptxAzure Data.pptx
Azure Data.pptx
FedoRam1
 
Big Data Ready Enterprise
Big Data Ready Enterprise Big Data Ready Enterprise
Big Data Ready Enterprise
DataWorks Summit/Hadoop Summit
 
Where to Begin? Application Portfolio Migration
Where to Begin? Application Portfolio MigrationWhere to Begin? Application Portfolio Migration
Where to Begin? Application Portfolio Migration
Amazon Web Services
 
(ATS4-DEV09) Visualizing SmartLab Data Using the Accelrys Enterprise Platform
(ATS4-DEV09) Visualizing SmartLab Data Using the Accelrys Enterprise Platform(ATS4-DEV09) Visualizing SmartLab Data Using the Accelrys Enterprise Platform
(ATS4-DEV09) Visualizing SmartLab Data Using the Accelrys Enterprise Platform
BIOVIA
 
Data Modernization_Harinath Susairaj.pptx
Data Modernization_Harinath Susairaj.pptxData Modernization_Harinath Susairaj.pptx
Data Modernization_Harinath Susairaj.pptx
ArunPandiyan890855
 
Azure Days 2019: Business Intelligence auf Azure (Marco Amhof & Yves Mauron)
Azure Days 2019: Business Intelligence auf Azure (Marco Amhof & Yves Mauron)Azure Days 2019: Business Intelligence auf Azure (Marco Amhof & Yves Mauron)
Azure Days 2019: Business Intelligence auf Azure (Marco Amhof & Yves Mauron)
Trivadis
 
SecureKloud_Corporate Deck.pdf
SecureKloud_Corporate Deck.pdfSecureKloud_Corporate Deck.pdf
SecureKloud_Corporate Deck.pdf
SrinivasMahankali3
 
SAP ASE In The Cloud
SAP ASE In The Cloud SAP ASE In The Cloud
SAP ASE In The Cloud
SAP Technology
 
Windowsazureplatform Overviewlatest
Windowsazureplatform OverviewlatestWindowsazureplatform Overviewlatest
Windowsazureplatform Overviewlatest
rajramab
 
Enterprise Cloud Data Platforms - with Microsoft Azure
Enterprise Cloud Data Platforms - with Microsoft AzureEnterprise Cloud Data Platforms - with Microsoft Azure
Enterprise Cloud Data Platforms - with Microsoft Azure
Khalid Salama
 
Informatica Cloud Summer 2016 Release Webinar Slides
Informatica Cloud Summer 2016 Release Webinar SlidesInformatica Cloud Summer 2016 Release Webinar Slides
Informatica Cloud Summer 2016 Release Webinar Slides
Informatica Cloud
 
Samuel Bayeta
Samuel BayetaSamuel Bayeta
Samuel Bayeta
Sam B
 
CData Power BI Connectors - MS Business Application Summit
CData Power BI Connectors - MS Business Application SummitCData Power BI Connectors - MS Business Application Summit
CData Power BI Connectors - MS Business Application Summit
Jerod Johnson
 
StreamCentral Technical Overview
StreamCentral Technical OverviewStreamCentral Technical Overview
StreamCentral Technical Overview
Raheel Retiwalla
 
Data & Analytics ReInvent Recap [AWS Basel Meetup - Jan 2023].pdf
Data & Analytics ReInvent Recap [AWS Basel Meetup - Jan 2023].pdfData & Analytics ReInvent Recap [AWS Basel Meetup - Jan 2023].pdf
Data & Analytics ReInvent Recap [AWS Basel Meetup - Jan 2023].pdf
Chris Bingham
 
Ipedo Company Overview
Ipedo Company OverviewIpedo Company Overview
Ipedo Company Overview
Tim_Matthews
 
Big SQL 3.0 - Fast and easy SQL on Hadoop
Big SQL 3.0 - Fast and easy SQL on HadoopBig SQL 3.0 - Fast and easy SQL on Hadoop
Big SQL 3.0 - Fast and easy SQL on Hadoop
Wilfried Hoge
 

Similar to 5. iED Cloud Services.pdf (20)

Single View of Well, Production and Assets
Single View of Well, Production and AssetsSingle View of Well, Production and Assets
Single View of Well, Production and Assets
 
Azure SQL Database Managed Instance - technical overview
Azure SQL Database Managed Instance - technical overviewAzure SQL Database Managed Instance - technical overview
Azure SQL Database Managed Instance - technical overview
 
Dev show september 8th 2020 power platform - not just a simple toy
Dev show september 8th 2020   power platform - not just a simple toyDev show september 8th 2020   power platform - not just a simple toy
Dev show september 8th 2020 power platform - not just a simple toy
 
Azure Data.pptx
Azure Data.pptxAzure Data.pptx
Azure Data.pptx
 
Big Data Ready Enterprise
Big Data Ready Enterprise Big Data Ready Enterprise
Big Data Ready Enterprise
 
Where to Begin? Application Portfolio Migration
Where to Begin? Application Portfolio MigrationWhere to Begin? Application Portfolio Migration
Where to Begin? Application Portfolio Migration
 
(ATS4-DEV09) Visualizing SmartLab Data Using the Accelrys Enterprise Platform
(ATS4-DEV09) Visualizing SmartLab Data Using the Accelrys Enterprise Platform(ATS4-DEV09) Visualizing SmartLab Data Using the Accelrys Enterprise Platform
(ATS4-DEV09) Visualizing SmartLab Data Using the Accelrys Enterprise Platform
 
Data Modernization_Harinath Susairaj.pptx
Data Modernization_Harinath Susairaj.pptxData Modernization_Harinath Susairaj.pptx
Data Modernization_Harinath Susairaj.pptx
 
Azure Days 2019: Business Intelligence auf Azure (Marco Amhof & Yves Mauron)
Azure Days 2019: Business Intelligence auf Azure (Marco Amhof & Yves Mauron)Azure Days 2019: Business Intelligence auf Azure (Marco Amhof & Yves Mauron)
Azure Days 2019: Business Intelligence auf Azure (Marco Amhof & Yves Mauron)
 
SecureKloud_Corporate Deck.pdf
SecureKloud_Corporate Deck.pdfSecureKloud_Corporate Deck.pdf
SecureKloud_Corporate Deck.pdf
 
SAP ASE In The Cloud
SAP ASE In The Cloud SAP ASE In The Cloud
SAP ASE In The Cloud
 
Windowsazureplatform Overviewlatest
Windowsazureplatform OverviewlatestWindowsazureplatform Overviewlatest
Windowsazureplatform Overviewlatest
 
Enterprise Cloud Data Platforms - with Microsoft Azure
Enterprise Cloud Data Platforms - with Microsoft AzureEnterprise Cloud Data Platforms - with Microsoft Azure
Enterprise Cloud Data Platforms - with Microsoft Azure
 
Informatica Cloud Summer 2016 Release Webinar Slides
Informatica Cloud Summer 2016 Release Webinar SlidesInformatica Cloud Summer 2016 Release Webinar Slides
Informatica Cloud Summer 2016 Release Webinar Slides
 
Samuel Bayeta
Samuel BayetaSamuel Bayeta
Samuel Bayeta
 
CData Power BI Connectors - MS Business Application Summit
CData Power BI Connectors - MS Business Application SummitCData Power BI Connectors - MS Business Application Summit
CData Power BI Connectors - MS Business Application Summit
 
StreamCentral Technical Overview
StreamCentral Technical OverviewStreamCentral Technical Overview
StreamCentral Technical Overview
 
Data & Analytics ReInvent Recap [AWS Basel Meetup - Jan 2023].pdf
Data & Analytics ReInvent Recap [AWS Basel Meetup - Jan 2023].pdfData & Analytics ReInvent Recap [AWS Basel Meetup - Jan 2023].pdf
Data & Analytics ReInvent Recap [AWS Basel Meetup - Jan 2023].pdf
 
Ipedo Company Overview
Ipedo Company OverviewIpedo Company Overview
Ipedo Company Overview
 
Big SQL 3.0 - Fast and easy SQL on Hadoop
Big SQL 3.0 - Fast and easy SQL on HadoopBig SQL 3.0 - Fast and easy SQL on Hadoop
Big SQL 3.0 - Fast and easy SQL on Hadoop
 

Recently uploaded

CAP Excel Formulas & Functions July - Copy (4).pdf
CAP Excel Formulas & Functions July - Copy (4).pdfCAP Excel Formulas & Functions July - Copy (4).pdf
CAP Excel Formulas & Functions July - Copy (4).pdf
frp60658
 
🔥Mature Women / Aunty Call Girl Chennai 💯Call Us 🔝 8094342248 🔝💃Top Class Cal...
🔥Mature Women / Aunty Call Girl Chennai 💯Call Us 🔝 8094342248 🔝💃Top Class Cal...🔥Mature Women / Aunty Call Girl Chennai 💯Call Us 🔝 8094342248 🔝💃Top Class Cal...
🔥Mature Women / Aunty Call Girl Chennai 💯Call Us 🔝 8094342248 🔝💃Top Class Cal...
shivangimorya083
 
Call Girls Hyderabad (india) ☎️ +91-7426014248 Hyderabad Call Girl
Call Girls Hyderabad  (india) ☎️ +91-7426014248 Hyderabad  Call GirlCall Girls Hyderabad  (india) ☎️ +91-7426014248 Hyderabad  Call Girl
Call Girls Hyderabad (india) ☎️ +91-7426014248 Hyderabad Call Girl
sapna sharmap11
 
Delhi Call Girls Karol Bagh 👉 9711199012 👈 unlimited short high profile full ...
Delhi Call Girls Karol Bagh 👉 9711199012 👈 unlimited short high profile full ...Delhi Call Girls Karol Bagh 👉 9711199012 👈 unlimited short high profile full ...
Delhi Call Girls Karol Bagh 👉 9711199012 👈 unlimited short high profile full ...
mona lisa $A12
 
🔥Call Girl Price Pune 💯Call Us 🔝 7014168258 🔝💃Independent Pune Escorts Servic...
🔥Call Girl Price Pune 💯Call Us 🔝 7014168258 🔝💃Independent Pune Escorts Servic...🔥Call Girl Price Pune 💯Call Us 🔝 7014168258 🔝💃Independent Pune Escorts Servic...
🔥Call Girl Price Pune 💯Call Us 🔝 7014168258 🔝💃Independent Pune Escorts Servic...
Ak47
 
A review of I_O behavior on Oracle database in ASM
A review of I_O behavior on Oracle database in ASMA review of I_O behavior on Oracle database in ASM
A review of I_O behavior on Oracle database in ASM
Alireza Kamrani
 
Product Cluster Analysis: Unveiling Hidden Customer Preferences
Product Cluster Analysis: Unveiling Hidden Customer PreferencesProduct Cluster Analysis: Unveiling Hidden Customer Preferences
Product Cluster Analysis: Unveiling Hidden Customer Preferences
Boston Institute of Analytics
 
machine learning notes by Andrew Ng and Tengyu Ma
machine learning notes by Andrew Ng and Tengyu Mamachine learning notes by Andrew Ng and Tengyu Ma
machine learning notes by Andrew Ng and Tengyu Ma
Vijayabaskar Uthirapathy
 
Direct Lake Deep Dive slides from Fabric Engineering Roadshow
Direct Lake Deep Dive slides from Fabric Engineering RoadshowDirect Lake Deep Dive slides from Fabric Engineering Roadshow
Direct Lake Deep Dive slides from Fabric Engineering Roadshow
Gabi Münster
 
9711199012⎷❤✨ Call Girls RK Puram Special Price with a special young
9711199012⎷❤✨ Call Girls RK Puram Special Price with a special young9711199012⎷❤✨ Call Girls RK Puram Special Price with a special young
9711199012⎷❤✨ Call Girls RK Puram Special Price with a special young
Ak47
 
Fabric Engineering Deep Dive Keynote from Fabric Engineering Roadshow
Fabric Engineering Deep Dive Keynote from Fabric Engineering RoadshowFabric Engineering Deep Dive Keynote from Fabric Engineering Roadshow
Fabric Engineering Deep Dive Keynote from Fabric Engineering Roadshow
Gabi Münster
 
一比一原版(heriotwatt学位证书)英国赫瑞瓦特大学毕业证如何办理
一比一原版(heriotwatt学位证书)英国赫瑞瓦特大学毕业证如何办理一比一原版(heriotwatt学位证书)英国赫瑞瓦特大学毕业证如何办理
一比一原版(heriotwatt学位证书)英国赫瑞瓦特大学毕业证如何办理
zoykygu
 
Call Girls Goa👉9024918724👉Low Rate Escorts in Goa 💃 Available 24/7
Call Girls Goa👉9024918724👉Low Rate Escorts in Goa 💃 Available 24/7Call Girls Goa👉9024918724👉Low Rate Escorts in Goa 💃 Available 24/7
Call Girls Goa👉9024918724👉Low Rate Escorts in Goa 💃 Available 24/7
nitachopra
 
Call Girls Hyderabad ❤️ 7339748667 ❤️ With No Advance Payment
Call Girls Hyderabad ❤️ 7339748667 ❤️ With No Advance PaymentCall Girls Hyderabad ❤️ 7339748667 ❤️ With No Advance Payment
Call Girls Hyderabad ❤️ 7339748667 ❤️ With No Advance Payment
prijesh mathew
 
Do People Really Know Their Fertility Intentions? Correspondence between Sel...
Do People Really Know Their Fertility Intentions?  Correspondence between Sel...Do People Really Know Their Fertility Intentions?  Correspondence between Sel...
Do People Really Know Their Fertility Intentions? Correspondence between Sel...
Xiao Xu
 
Optimizing Feldera: Integrating Advanced UDFs and Enhanced SQL Functionality ...
Optimizing Feldera: Integrating Advanced UDFs and Enhanced SQL Functionality ...Optimizing Feldera: Integrating Advanced UDFs and Enhanced SQL Functionality ...
Optimizing Feldera: Integrating Advanced UDFs and Enhanced SQL Functionality ...
mparmparousiskostas
 
Ahmedabad Call Girls 7339748667 With Free Home Delivery At Your Door
Ahmedabad Call Girls 7339748667 With Free Home Delivery At Your DoorAhmedabad Call Girls 7339748667 With Free Home Delivery At Your Door
Ahmedabad Call Girls 7339748667 With Free Home Delivery At Your Door
Russian Escorts in Delhi 9711199171 with low rate Book online
 
Salesforce AI + Data Community Tour Slides - Canarias
Salesforce AI + Data Community Tour Slides - CanariasSalesforce AI + Data Community Tour Slides - Canarias
Salesforce AI + Data Community Tour Slides - Canarias
davidpietrzykowski1
 
Difference in Differences - Does Strict Speed Limit Restrictions Reduce Road ...
Difference in Differences - Does Strict Speed Limit Restrictions Reduce Road ...Difference in Differences - Does Strict Speed Limit Restrictions Reduce Road ...
Difference in Differences - Does Strict Speed Limit Restrictions Reduce Road ...
ThinkInnovation
 
❻❸❼⓿❽❻❷⓿⓿❼KALYAN MATKA CHART FINAL OPEN JODI PANNA FIXXX DPBOSS MATKA RESULT ...
❻❸❼⓿❽❻❷⓿⓿❼KALYAN MATKA CHART FINAL OPEN JODI PANNA FIXXX DPBOSS MATKA RESULT ...❻❸❼⓿❽❻❷⓿⓿❼KALYAN MATKA CHART FINAL OPEN JODI PANNA FIXXX DPBOSS MATKA RESULT ...
❻❸❼⓿❽❻❷⓿⓿❼KALYAN MATKA CHART FINAL OPEN JODI PANNA FIXXX DPBOSS MATKA RESULT ...
#kalyanmatkaresult #dpboss #kalyanmatka #satta #matka #sattamatka
 

Recently uploaded (20)

CAP Excel Formulas & Functions July - Copy (4).pdf
CAP Excel Formulas & Functions July - Copy (4).pdfCAP Excel Formulas & Functions July - Copy (4).pdf
CAP Excel Formulas & Functions July - Copy (4).pdf
 
🔥Mature Women / Aunty Call Girl Chennai 💯Call Us 🔝 8094342248 🔝💃Top Class Cal...
🔥Mature Women / Aunty Call Girl Chennai 💯Call Us 🔝 8094342248 🔝💃Top Class Cal...🔥Mature Women / Aunty Call Girl Chennai 💯Call Us 🔝 8094342248 🔝💃Top Class Cal...
🔥Mature Women / Aunty Call Girl Chennai 💯Call Us 🔝 8094342248 🔝💃Top Class Cal...
 
Call Girls Hyderabad (india) ☎️ +91-7426014248 Hyderabad Call Girl
Call Girls Hyderabad  (india) ☎️ +91-7426014248 Hyderabad  Call GirlCall Girls Hyderabad  (india) ☎️ +91-7426014248 Hyderabad  Call Girl
Call Girls Hyderabad (india) ☎️ +91-7426014248 Hyderabad Call Girl
 
Delhi Call Girls Karol Bagh 👉 9711199012 👈 unlimited short high profile full ...
Delhi Call Girls Karol Bagh 👉 9711199012 👈 unlimited short high profile full ...Delhi Call Girls Karol Bagh 👉 9711199012 👈 unlimited short high profile full ...
Delhi Call Girls Karol Bagh 👉 9711199012 👈 unlimited short high profile full ...
 
🔥Call Girl Price Pune 💯Call Us 🔝 7014168258 🔝💃Independent Pune Escorts Servic...
🔥Call Girl Price Pune 💯Call Us 🔝 7014168258 🔝💃Independent Pune Escorts Servic...🔥Call Girl Price Pune 💯Call Us 🔝 7014168258 🔝💃Independent Pune Escorts Servic...
🔥Call Girl Price Pune 💯Call Us 🔝 7014168258 🔝💃Independent Pune Escorts Servic...
 
A review of I_O behavior on Oracle database in ASM
A review of I_O behavior on Oracle database in ASMA review of I_O behavior on Oracle database in ASM
A review of I_O behavior on Oracle database in ASM
 
Product Cluster Analysis: Unveiling Hidden Customer Preferences
Product Cluster Analysis: Unveiling Hidden Customer PreferencesProduct Cluster Analysis: Unveiling Hidden Customer Preferences
Product Cluster Analysis: Unveiling Hidden Customer Preferences
 
machine learning notes by Andrew Ng and Tengyu Ma
machine learning notes by Andrew Ng and Tengyu Mamachine learning notes by Andrew Ng and Tengyu Ma
machine learning notes by Andrew Ng and Tengyu Ma
 
Direct Lake Deep Dive slides from Fabric Engineering Roadshow
Direct Lake Deep Dive slides from Fabric Engineering RoadshowDirect Lake Deep Dive slides from Fabric Engineering Roadshow
Direct Lake Deep Dive slides from Fabric Engineering Roadshow
 
9711199012⎷❤✨ Call Girls RK Puram Special Price with a special young
9711199012⎷❤✨ Call Girls RK Puram Special Price with a special young9711199012⎷❤✨ Call Girls RK Puram Special Price with a special young
9711199012⎷❤✨ Call Girls RK Puram Special Price with a special young
 
Fabric Engineering Deep Dive Keynote from Fabric Engineering Roadshow
Fabric Engineering Deep Dive Keynote from Fabric Engineering RoadshowFabric Engineering Deep Dive Keynote from Fabric Engineering Roadshow
Fabric Engineering Deep Dive Keynote from Fabric Engineering Roadshow
 
一比一原版(heriotwatt学位证书)英国赫瑞瓦特大学毕业证如何办理
一比一原版(heriotwatt学位证书)英国赫瑞瓦特大学毕业证如何办理一比一原版(heriotwatt学位证书)英国赫瑞瓦特大学毕业证如何办理
一比一原版(heriotwatt学位证书)英国赫瑞瓦特大学毕业证如何办理
 
Call Girls Goa👉9024918724👉Low Rate Escorts in Goa 💃 Available 24/7
Call Girls Goa👉9024918724👉Low Rate Escorts in Goa 💃 Available 24/7Call Girls Goa👉9024918724👉Low Rate Escorts in Goa 💃 Available 24/7
Call Girls Goa👉9024918724👉Low Rate Escorts in Goa 💃 Available 24/7
 
Call Girls Hyderabad ❤️ 7339748667 ❤️ With No Advance Payment
Call Girls Hyderabad ❤️ 7339748667 ❤️ With No Advance PaymentCall Girls Hyderabad ❤️ 7339748667 ❤️ With No Advance Payment
Call Girls Hyderabad ❤️ 7339748667 ❤️ With No Advance Payment
 
Do People Really Know Their Fertility Intentions? Correspondence between Sel...
Do People Really Know Their Fertility Intentions?  Correspondence between Sel...Do People Really Know Their Fertility Intentions?  Correspondence between Sel...
Do People Really Know Their Fertility Intentions? Correspondence between Sel...
 
Optimizing Feldera: Integrating Advanced UDFs and Enhanced SQL Functionality ...
Optimizing Feldera: Integrating Advanced UDFs and Enhanced SQL Functionality ...Optimizing Feldera: Integrating Advanced UDFs and Enhanced SQL Functionality ...
Optimizing Feldera: Integrating Advanced UDFs and Enhanced SQL Functionality ...
 
Ahmedabad Call Girls 7339748667 With Free Home Delivery At Your Door
Ahmedabad Call Girls 7339748667 With Free Home Delivery At Your DoorAhmedabad Call Girls 7339748667 With Free Home Delivery At Your Door
Ahmedabad Call Girls 7339748667 With Free Home Delivery At Your Door
 
Salesforce AI + Data Community Tour Slides - Canarias
Salesforce AI + Data Community Tour Slides - CanariasSalesforce AI + Data Community Tour Slides - Canarias
Salesforce AI + Data Community Tour Slides - Canarias
 
Difference in Differences - Does Strict Speed Limit Restrictions Reduce Road ...
Difference in Differences - Does Strict Speed Limit Restrictions Reduce Road ...Difference in Differences - Does Strict Speed Limit Restrictions Reduce Road ...
Difference in Differences - Does Strict Speed Limit Restrictions Reduce Road ...
 
❻❸❼⓿❽❻❷⓿⓿❼KALYAN MATKA CHART FINAL OPEN JODI PANNA FIXXX DPBOSS MATKA RESULT ...
❻❸❼⓿❽❻❷⓿⓿❼KALYAN MATKA CHART FINAL OPEN JODI PANNA FIXXX DPBOSS MATKA RESULT ...❻❸❼⓿❽❻❷⓿⓿❼KALYAN MATKA CHART FINAL OPEN JODI PANNA FIXXX DPBOSS MATKA RESULT ...
❻❸❼⓿❽❻❷⓿⓿❼KALYAN MATKA CHART FINAL OPEN JODI PANNA FIXXX DPBOSS MATKA RESULT ...
 

5. iED Cloud Services.pdf

  • 1. © 2018 iED 1 INTEGRATE. ENABLE. DRIVE.
  • 2. © 2018 iED 2 Agenda Industry Challenges Reusable Assets 1 2 4 5 6 Solution Architecture 7 8 3 Platform Supported DevOps Value Proposition iED
  • 3. © 2018 iED 3 Strategize, Assess & Plan Design, Build & Validate Run & Optimize CLOUD OPERATIONS SERVICES DATA LAKE CONSULTING SERVICES DATA MIGRATION, DATA LAKE DESIGN & BUILD iED | Your Trusted Technology Partner • Cloud strategy • Fit Gap Assessment • Migration Strategy • Data Lake & Analytical Roadmap • Data Lake Design & Build on Cloud • Data Migration, Transformation, & Validation • Data Security Implementation • Data Management & Monitoring • Service Management • Security and Identity Management • Capacity Planning and Optimization • Performance and Troubleshooting iED was founded after several decades of the learning by industry leaders to create production grade data lakes that accelerate advanced analytics and machine learning to play a critical role in transforming enterprises.
  • 4. © 2018 iED 4 Industry Challenges & Highlights Data certification Inadequate “CloudFirst” focus Rapidly evolving technology choices Bringing agility and scalability Fit-gap analysis leading to technology selection Rule driven data quality engine • Pre- orchestrated ingestion, storage and consumption engine • Automated monitoring and scheduling • Configurable, functional, and business rules • Functional and technical validation • Layer based balancing • Responsive data architecture • Intelligent data management • Delivery at scale Current Challenges Our Solution Key Highlights • Searchable meta data and lineage • Enterprise semantic layer • Domains: Multiple Self service * Time to value Pre-built domain taxonomy Automation and asset based implementation • Integration Framework • Audit, Balancing & Control • Data lineage * • Continuous integration/deployment * Future road map
  • 5. © 2018 iED About VISVA  Quickly and easily set up massive ingestion with an intuitive GUI  Metadata driven data ingestion  Maximize throughput using parallel threading  Web-based control features including restart-ability, throughput monitoring and notifications Ingestion Validation Transformation Consumption  MD5 Checksum based validations  Multi-level data validation report (row-count, row-level, cell level)  Optimized data comparator using custom code (spark, native)  Metadata validation  Data Profiling and Standardization  Dynamic creation of Logical views  Point In Time view of source data  Support for multiple file formats – AVRO, ORC, etc.  Analytical data sets  Augmented Data Discovery  Enterprise semantic layer  Data Visualization and self service 5 “VISVA is iED’s Cloud Data platform that facilitates modernization of enterprise Data Assets to maximize performance and reduce cost. VISVA frameworks brings 4x performance improvement and more than 60% effort savings.”
  • 6. © 2018 iED VISVA | Implementation 6 Platform supported Compute Storage Sources HDInsight Hadoop Data Lake Store Mainframe Databricks SQL Database File Systems Data Lake Analytics SQL Data Warehouse RDBMS Native Spark Blob Cloud Azure Batch Analysis Services Web Hadoop Salesforce Packages Basic (option to select one of the four below) Advanced Premium Ingestion    Transformation   Validation   Consumption  70% Automated Data Ingestion 50% Faster Time to Deployment 60% Automated Schema Generation 99% Data Accuracy
  • 7. © 2018 iED 7 Solution Architecture
  • 8. © 2018 iED 8 Data Accquisiton Perform Data Standardization to create staging layer Landing Layer Authentication, Authorization, Accounting & Data Protection Data Processing Layer AES-256 Consumption Layer Source Layer Appliance RDBMS Flat Files Audit Balance and Control Framework Monitoring and Administration Active Copy Layer Staging Layer Batch Data Consumption Azure Data Lake Store Azure Active Directory Load Curated data to ADW using Polybase Azure Monitor Express Route AES-256 AES-256 Azure Data Lake Store Azure Data Lake Store Azure Data Lake Store AES-256 Curation Layer Create point in time view of data in active copy layer Apply ETL transformation to created curated layer Web Real Time Express Route ADF V2 Integration Framework 1 2 3 4 5 6 7 Data Comparator Tool Azure SQL DB Azure SQL Data Warehouse 8 VISVA | Solution Architecture Excel BI Power BI SSRS Data Zen SharePoint BI Visualization
  • 9. © 2018 iED 9 VISVA | Architecture Highlights • Data Integration Framework for extracting and loading large volume of data from a variety of data sources onto Azure/AWS • Dynamic Metadata Synchronization between On-premise and Azure/AWS • Audit, Balance and Control - framework is capable of providing detailed Audit Reports including process ran, success rates and failures along with functional/business rule validation • Monitoring & Scheduling - The Monitoring Workbench provides the facility to automate the workflows involved in the enterprise operational process through its scheduler
  • 11. © 2018 iED 11 Azure WebApp - WebJob 1 CDI Framework Configuration Database Azure WebJob to execute the Integration Framework Executable Integration Framework reads the Configuration Database for Data movement and Orchestration configurations 2 3 Integration Framework generates the ADF V2 pipelines for Data Movement and Orchestration activities Data Movement ADF V2 Pipelines Workflow orchestration activity Transform Data in Active Copy Zone using Databricks activity Datahub Landing Zone ADLS On-Premise Sources Netezza, DB2, SQL Server, Flat Files, Web Datahub Staging Zone ADLS Datahub Active Copy Zone ADLS a b c Provisioning of Sources in Azure Data Lake Store 4 Apply ETL transformation and build data marts 5 Databricks Spark Activity Datahub Curation Zone ADLS Datahub Consumption Zone 6 a b Polybase Activity c VISVA | Integration Framework Metadata Driven Integration Framework to Provision a Data Source from On-Premise to Azure/AWS
  • 12. © 2018 iED 12 Asp .NET Azure Web App Custom Scheduler Azure Automation Runbook Runbook polls to check if schedule has arrived for a workflow to execute Custom Job Status Portal to monitor the status of workflow Azure Data Factory V2 ZEA Pipeline Runbook triggers the ADF V2 pipelines Validate the workflow schedule details from Schedule DB Workflow Schedule Table Workflow Status Table ADF Pipeline Status Table ADF Activity Status Table Production Support Team Support team generates Reports/metric by querying operational Audit tables Customer reviews Status report Support team publish health report to customer 1 2 3 4 ADF V2 pipeline executes workflow and updates the operational audit details in Audit Database Operational Audit Database 7 8 9 Functional Rule Database Transformed and prepared data is validated against functional rules defined in Rule database Functional Reconciliation Framework 5 6 Functional Reconciliation activity is triggered post workflow execution VISVA | Audit Balance & Control Automated Configurable ABC Framework which governs and certifies the data across the ecosystem
  • 13. © 2018 iED 13 Integration framework reads the config. database and generates data lineage pipelines Data Engineers ETL workflow captured in SQL DB template Integration Framework configuration database SQL DB project template in Visual Studio ADF V2 pipelines are deployed in to Data Factory VSTS GIT VSTS CI/CD Pipeline Visual Studio Team Services Azure Data Factory V2 Pipeline CI/CD process deploys the SQL DB project changes to config. database 1 6 5 4 3 2 Integration Framework ETL workflow SQL DB project checked-in to VSTS GIT Azure Cosmos Graph DB ADF V2 pipelines updates the Graph DB with lineage details 7 Data Lineage Data Lineage captured through custom API or enterprise lineage tools VISVA | Data Lineage (Future Roadmap)
  • 17. © 2018 iED 17 Plan + Track Release Defect Management Track Progress Manage work Plan work Project Starts Automation functional testing environment Automation Integration testing environment Pre-production Environment Staging environment Track defect Manage defect Defect matrix Build Version control Unit testing Write code Develop + Test Report status Defect Management Track Defects, tasks, and blocking issues Project Management Project plan by creating a backlog of user stories that represent the work Release Automation Automated release pipelines in which you can reliably test and release on much shorter cycles. Development Develop, unit test and build automation VISVA |Continuous Integration/Deployment
  翻译: