IoT Meets Big Data: The Opportunities and Challenges by Syed Hoda of ParStream

The IoT Inc Business Meetup Silicon Valley
Meeting 8
April 2015 Bruce Sinclair (Organizer): bruce@iot-inc.com

Target of Meetup
For business people selling products and services into IoT
but of course everyone else is welcome: techies, end-users, …
Focus of presentations and discussions:
© Iot-Inc. 2015

Looking for help
Reviews
• Help attract great speakers and members
• If you’ve been attending for a while and like the
group, go to homepage to leave a review
Help for our sponsor
• Need to justify their continued support
• Plug and Play are looking to partner with
companies big and small
Become a sponsor
Suggest locations to hold future meetings
© Iot-Inc. 2015

Notes
Presentation, video and notes for today’s meeting will be sent in
one week
© Iot-Inc. 2014
Next Meeting
Bruce Sinclair, President at Iot-Inc
Introduction to the Business of the Internet of Things

Analytics Built for IoT
IoT Meets Big Data:
The Opportunities and Challenges
Syed Hoda
Chief Marketing Officer
@shoda

#1
Big Data
Startup
The Industry’s Leading IoT Analytics Platform Company
! Massive volumes of data ! High-velocity data
! Edge analytics ! Real-time insights
Cisco Entrepreneurs in Residence 2014
IoT
Excellence
Award

www.parstream.com @parstream

3
Have you heard
about this IoT thing?

4
IoT is top-of-mind with CEO’s, CIO’s, and VC’s
Top 10 Strategic Technology Trends for 2015
1.  Computing Everywhere
2.  Internet of Things
3.  3-D Printing
4.  Advance, Pervasive Analytics
5.  Context-Rich Systems
6.  Smart Machines
7.  Cloud Computing
8.  Software Defined Infrastructure
9.  Web-scale IT
10.  Risk-Based Security
2015 Tech Predictions
1.  Digital transformation
2.  Internet of Things
3.  Convergence of big data with consumer data
4.  Hybrid cloud
5.  Collaboration
6.  Predictive analytics will lead big data
7.  Mobile wearable technology
8.  A Platform and orchestration is needed
9.  Networked Economy
10.  The end of apps
In 2014, investors contributed over $300 million
in 97 venture rounds for IoT startups
VCs Look To The Future As IoT Investments Soar

5
The massive size and growth of IoT
IoT
Market Size
(by 2025)
Connected
Devices
(by 2020)
Data
Growth
(2013 vs 2020)
$7.1T
$6.1T
$14.4T
26B
32B
50B
4.4ZB 44.4ZB
Total Data
10x
.09ZB 4.4ZB
IoT Data
49x

6
BIG data is outpacing Moore’s Law!
Data
Growth
(2013 vs 2020)
4.4ZB 44.4ZB
.09ZB 4.4ZB
Total Data
IoT Data
10x
49x
The Opportunity
AND
The Challenge

Analytics drives business value in IoT
“The value of IoT is in the data. The quicker
enterprises can start analyzing their data the more
business value they can derive.”
June 2014
Analytics have been transformative in wide areas
of customer and product service. Sensor enabled
industrial analytic applications are the next frontier
July 2014
“Analytics accelerates IoT adoption.
…data analytics is the most important factor to
increase the benefits of IoT”
John Chambers, October 2014

8
Just do it! Right?
Well, not so fast.

Market Pulse: Global IoT/Big Data Survey
-  Global/cross-industry survey
-  Cross functional participants
-  50/50 mix of business and
technology leaders
-  Various stages of IoT
experience and progress
-  Focus on the use and value of
data in IoT initiatives
Full report at:
sites.parstream.com/parstream-iot-
survey-whitepaper

Only a third have quantifiable success metrics
33% Have quantifiable metrics to track success
38% “Learning and exploring” is the objective
29% Have document goals, but difficult to quantify

96% have faced challenges with their IoT project
58% Business process/policy (e.g. privacy)
51% User adoption of new technology
41% Timely collection and analysis of data
40% Sensors or devices
4% Have not faced challenges

Challenges being faced at all stages of the data
collection and analysis process
44% Too much data to analyze effectively
36% Difficult to capture useful data
30% Analysis capabilities are not flexible or aligned
27% Not sure what questions to ask
26% Data is analyzed too slowly to be useful

Only 8% making full use of their IoT data
8% Fully capture and analyze data in a timely fashion
59% Do some analytics, but need to improve
17% Capture and store IoT data, but don’t/can’t analyze it
16% Don’t store IoT data

92% would see benefits by more effectively
capturing and storing IoT data
70% Make better, more meaningful decisions
53% Make decisions faster
27% Make more decisions
8% No benefits

Business owners more likely to see ROI increase
through faster, more flexible analytics
Business
58% Significant increase in ROI 28%
Technology
34% Slight increase in ROI 57%
8% Minimal Impact on ROI 15%

Better IoT data collection and analysis would deliver more value
86% Would increase the ROI of their IoT investment
IoT not delivering full potential because of data challenges
8% Fully capture and analyze IoT data in a timely fashion
IoT projects vary widely – but all have challenges
96% Faced project challenges (#1 process, #2 users, #3 data)
Survey Summary: Three key insights

17
Other than that Mrs. Lincoln,
how was the play?

Imagine a world…
Where IoT analytics enable an energy company to…
30TB
Analyze Data
in Real-time
15%
Increase
Efficiency
$18K/hr; $158M/yr
Generate Operational/
Economic Benefits
(20,000 Wind Turbines; 10 GW Capacity; .3 Capacity Factor; $40/MW-hour)
18

IoT analytics has a set of distinct requirements
19
Big Data
Data is growing faster and bigger
because of number of sensors
10B+ rows
5TB+
Fast Data
Data streamed from sensors
requires fast ingestion
1M+ rows
per sec
Edge Analytics
IoT data is mostly generated
at the ‘Edges’ of the network
100+
Locations
Real-Time Insights
Use cases require near
real time analytics
<1 sec query
response
time

IoT analytics has a set of distinct requirements.
20
Big Data
Data is growing faster and bigger
because of number of sensors
10B+ rows
5TB+
Wind turbine: 100 turbines x 100M rows per year
Race car: 400M records / day x 365 days test drive
Telco: 1.000 cells x 1.000 rows / sec x 1 days - wow
Traffic analysis: 60M cars x 1 read / min x 365 days
Oil rig: 1 rig = 8 billion records / day (not verified)
Fast Data
Data streamed from sensors
requires fast ingestion
1M+ rows
per sec
Network monitoring: 1M rows per sec per cell
Asset monitoring: 60M cars x 1 reading per minute
Airplane monitoring: 4 turbines x 3k sensors x 100Hz
Oil exploration: 10.000 wells x 100 sensors x 1Hz
Oil rig: 1 drilling rig x 10.000 sensors x avg 100Hz
Edge Analytics
IoT data is mostly generated
at the ‘Edges’ of the network
100+
Locations
Manufacturing: 300.000 plants in US (2012)
Cars / ships / airplanes: >1 billion world wide
Telco: 190.000 cell towers in US (2013)
Oil: 950.000 wells worldwide; 500.000 in US
Mobile advertising: de-central adserving / monitoring
Real-Time Insights
Use cases require near
real time analytics
<1 sec query
response
time
Dashboarding: real-time visualization, many queries
Network monitoring: root cause analysis, optimization
Asset monitoring: conditional monitoring, safety
Security: anomalie detection, building safety
Traffic: location aware recommendations

Existing products don’t fulfill IoT requirements
21
Product
Requirements
Columnar
Databases
HP Vertica,
Redshift
Row-based
Databases
Oracle,
Informix...
Value
Stores
Cassandra,
MongoDB
Hadoop
Batch
Cloudera,
Hortonworks
Hadoop
Streaming
Spark / Shark
Storm
BIG DATA
Capacity –
FAST DATA
Import – –
EDGE
Analytics Capability – – – – –
REAL TIME
Insights – – – – –
INTEGRATED
Platform – – –
IoT DATA
Storage Structure – – –
See details in backup

ParStream has the fastest query response times
22
Environment: Single EC2 XL node with 15 GB RAM, 2 TB disk on Amazon AWS.
OTP data set with 150 Million records. Query set based on customer use-cases.
RedShift
1
second
10
seconds
22
seconds
31
seconds
38
seconds
98
seconds

Use-cases should drive technology decisions
23
< 1..10 ms
1 min
1 hr
10..100 ms
1 sec
10 min
4 hrs
ResponseTime
Big Data
Massively parallel (MPP)
Real-Time
Hadoop / Cassandra / Impala
OLTP
Reporting
In-Memory DB
Gigabyte Terabyte Petabyte
OLAP
Batch-Analytics
Real-Time IoT
Analytics
Stream-Analytics
Operations
Analytics
Complex Event
Processing

ParStream is integrated with leading IoT solutions
24
Custom Apps DATAWATCH

AnalyticsVisualization
Data
Collection

“Doing” IoT
26
1.  Who’s in charge here?: The need for a Chief IoT Officer!
2.  Data = Competitive Advantage: Analyze more and more often
3.  Follow the money: Where/how can IoT data monetized?
4.  Use cases drive technology decisions: Information > IT
5.  Rational experimentation: IoT 2015 = eBusiness 1999

“Selling” IoT
27
1.  Follow the money: Help customers make money or save money!
2.  Agile product roadmaps: Sense and respond faster!
3.  “Sell” to Business + IT: IoT has a complex, evolving “org chart”
4.  Whole offer: Build/buy/partner to create what customers want
5.  The power of platforms: Metcalf’s Law = Relevance

Thank you!
Syed Hoda
Chief Marketing Officer
@shoda

Backup Slides

ParStream is the only solution for IoT analytics requirements
30
Customer Applications and Visualization Tools
IoT Data Collection Platforms Enterprise Data Sources
ParStream DB
Geo-
Distributed
Analytics
Alarm +
Action
Time
Series
Advanced
Analytics

ParStream is uniquely positioned for real-time IoT analytics
31
REAL-TIME
IMPORT
REAL-TIME
QUERYING
FLEXIBLE
ANALYTICS
Small Form Factor / Low TCO
BillionsofRecords
Thousands of Columns

ParStream’s patented technology provides a
competitive advantage
1
2
3 Lockless architecture
Enables ultra-fast query and
data import performance
Massive parallel processing
Delivers linear scalability and
high query throughput
4 Small footprint
Enables analytics at the edge
with a low TCO
High Performance
Compressed Indexes
Provide ultra-high query
performance
SQL API / JDBC /ODBC C++ UDx API
Real-Time Analytics Engine
In-Memory and
Disk Technology
Multi-Dimensional
Partitioning
Massively Parallel
Processing (MPP)
Shared Nothing
Architecture
3rd generation Columnar Storage
High Speed Parallel Loader with Low Latency
High
Performance
Compressed
Index (HPCI)
32

Edge analytics delivers real-time insights by
minimizing network traffic
33
20 Billion Rows
40 records found
ParStream
ParStream Geo-Distributed Server
3
rows
14
rows
5
rows
12
rows
6
rows
ParStream ParStream ParStream ParStream
Keith Nosbusch, CEO Rockwell Automation
“remote monitoring and diagnostics of physical assets such as liquid natural gas terminals… made possible by enabling
analytics at the source of data…will increase speed to insights and also alleviate the need to transport massive
amounts of data across the network”
40 Rows
ApplicationApplication
Centralized
storage
40 records found
4B
rows
4B
rows
4B
rows
4B
rows
4B
rows
Traditional Analytics Edge Analytics
20B
rows

ParStream introduces EdgeAnalyticsBox
The industry’s first appliance built for edge analytics/GDA
•  Specifically designed to enable edge analytics (Geo-Distributed Analytics).
•  Ruggedized for use in real-world edge analytics applications such as oil/drilling
sites, cell phone towers, wind farms, etc.
•  Pre-loaded and tested with ParStream software.
•  Technical Specs: Intel Core i5/i7 processor, 8-16 GB RAM and 64-128GB SSD
•  EdgeAnalyticsBox provides customers with the convenience of a one-stop shop
for the their edge analytics needs, however, customers can run GDA on any
standard hardware with certain processing and storage requirements.
34
New Product of the Week

Industry-leading Product Recognition
35
#1
Big Data
Startup
Cisco Entrepreneurs in Residence 2014
IoT
Excellence
Award

ParStream is the most
reliable System in our Data
Center
CTO, etracker
"ParStream’s ability to
analyze terabytes of data
with sub-second response
time helps us generate
significant value."
President, Envision Energy
ParStream enabled us to
scale internationally - TCO
is much lower than with
Hadoop
VP Eng, Searchmetrics

Demo: Sensor Analytics for Real-time Environmental Compliance
36

The key to generating value from IoT data:
Actionable Insights
Data
REAL-TIME DATA INGESTION + IMMEDIATE QUERIES
= ACTIONABLE / TIMELY INSIGHTS
Action
Devices
Devices
Devices
Rules or On-
demand Insights
Aggregation

IoT Meets Big Data: The Opportunities and Challenges by Syed Hoda of ParStream

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to IoT Meets Big Data: The Opportunities and Challenges by Syed Hoda of ParStream

Similar to IoT Meets Big Data: The Opportunities and Challenges by Syed Hoda of ParStream (20)

Recently uploaded

Recently uploaded (20)

IoT Meets Big Data: The Opportunities and Challenges by Syed Hoda of ParStream