Presto: Distributed sql query engine

Problem to solve
 Huge production of data.
 As data is growing enormously to the point of peta bytes
, querying the database has become a big issue.
 So we should be able to run more interactive queries and get
results faster .

Introduction
 Presto is a open source distributed sql query engine.
 For running queries against of all sizes ranging from
gigabytes to petabytes .
 It supports ANSI SQL ,including complex
queries,aggresgations,joins and window functions .
 It is implemented in java.

Architecture Explanation
 Client sends sql to presto coordinator.
 Coordinator parses ,analyzes and plans the query execution.
 The scheduler wires together the execution pipeline ,assigns
work to nodes closest to data and monitors the progress.
 The client pulls the data from output stage which in turn pulls
data from underlying stages.

Hive/Mapreduce Execution model
 Hive translates queries into multiple stage of mapreduce
tasks and execute them one after the other.
 Each task reads input from disk and writes intermediate
output back to disk.

Presto Execution
 Presto engine does not use Mapreduce.
 It employs a custom query and execution engine with
operators designed to support sql semantics.
 Processing is in memory and pipelined across the network
between stages which avoids unnecessary I/O and
associated latency overhead.
 Pipelined execution model runs multiple stages at once and
streams data from one stage to next as it becomes available
which reduces end-to-end latency

Note
 Presto dynamically compiles certain portions of query plan to
byte code which lets JVM optimize and generate native
machine code.

Extensibility
 Presto was designed with a simple storage abstraction that
makes its easy to provide sql query capability against
disparate data sources.
 Connectors only need to provide interfaces for fetching meta
data, getting data locations and accessing data itself.

Limitations
 Size limitation on the join tables and cardinality of unique
groups.
 Lacks the ability to write output back to tables. Currently
query results are streamed to client.

Presto developers claim:
 Presto is 10x better than hive/Mapreduce in terms of cpu
efficiency and latency for most queries.
 Supports ANSI sql, including joins, left/right outer
joins,subqueries,most of the common aggregate and scalar
functions, including approximate distinct counts,
approximate percentiles

Presto: Distributed sql query engine

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (9)

Similar to Presto: Distributed sql query engine

Similar to Presto: Distributed sql query engine (20)

Recently uploaded

Recently uploaded (20)

Presto: Distributed sql query engine