尊敬的 微信汇率:1円 ≈ 0.046166 元 支付宝汇率:1円 ≈ 0.046257元 [退出登录]
SlideShare a Scribd company logo
Parallel Computing

Concepts and Terminologies
Terminologies
• Task
   – A logically discrete section of computational work. A task is
     typically a program or program-like set of instructions that is
     executed by a processor.
• Parallel Task
   – A task that can be executed by multiple processors safely (yields
     correct results)
• Serial Execution
   – Execution of a program sequentially, one statement at a time. In
     the simplest sense, this is what happens on a one processor
     machine. However, virtually all parallel tasks will have sections of
     a parallel program that must be executed serially.
• Parallel Execution
   – Execution of a program by more than one task, with each task
     being able to execute the same or different statement at the same
     moment in time.
Terminologies
• Shared Memory
   – From a strictly hardware point of view, describes a computer
     architecture where all processors have direct (usually bus based)
     access to common physical memory. In a programming sense, it
     describes a model where parallel tasks all have the same "picture"
     of memory and can directly address and access the same logical
     memory locations regardless of where the physical memory
     actually exists.
• Distributed Memory
   – In hardware, refers to network based memory access for physical
     memory that is not common. As a programming model, tasks can
     only logically "see" local machine memory and must use
     communications to access memory on other machines where other
     tasks are executing
Terminologies
• Communications
   – Parallel tasks typically need to exchange data. There are several
     ways this can be accomplished, such as through a shared memory
     bus or over a network, however the actual event of data exchange
     is commonly referred to as communications regardless of the
     method employed.
• Synchronization
   – The coordination of parallel tasks in real time, very often
     associated with communications. Often implemented by
     establishing a synchronization point within an application where a
     task may not proceed further until another task(s) reaches the same
     or logically equivalent point. Synchronization usually involves
     waiting by at least one task, and can therefore cause a parallel
     application's wall clock execution time to increase
Terminologies
• Granularity
  – In parallel computing, granularity is a qualitative
    measure of the ratio of computation to communication.
     • Coarse: relatively large amounts of computational work are
       done between communication events
     • Fine: relatively small amounts of computational work are
       done between communication events
• Observed Speedup
  – Observed speedup of a code which has been
    parallelized, defined as: wall-clock time of
    serial execution / wall-clock time of
    parallel execution
  – One of the simplest and most widely used indicators for
von Neumann Architecture
• For over 40 years, virtually all computers have
  followed a common machine model known as the
  von Neumann computer. Named after the
  Hungarian mathematician John von Neumann.
• A von Neumann computer uses the stored-
  program concept. The CPU executes a stored
  program that specifies a sequence of read and
  write operations on the memory.
von Neumann Architecture
            •   Memory is used to store both
                program and data instructions
            •   Program instructions are coded
                data which tell the computer to
                do something
            •   Data is simply information to
                be used by the program
            •   A central processing unit
                (CPU) gets instructions and/or
                data from memory, decodes the
                instructions and then
                sequentially performs them
Flynn's Classical Taxonomy
• There are different ways to classify parallel
  computers. One of the more widely used
  classifications, in use since 1966, is called Flynn's
  Taxonomy.
• Flynn's taxonomy distinguishes multi-processor
  computer architectures according to how they can
  be classified along the two independent
  dimensions of Instruction and Data. Each of these
  dimensions can have only one of two possible
  states: Single or Multiple.
Flynn's Classical Taxonomy

• The matrix below defines the 4 possible
  classifications according to Flynn.


                  •S I S D                            •S I M D
      •Single Instruction, Single Data   •Single Instruction, Multiple Data


                •M I S D                            •M I M D
       •Multiple Instruction, Single      •Multiple Instruction, Multiple
                   Data                                Data
Single Instruction, Single Data (SISD)
• A serial (non-parallel) computer
• Single instruction: only one instruction
  stream is being acted on by the CPU
   during any one clock cycle
• Single data: only one data stream is
  being used as input during any one
  clock cycle
• Deterministic execution
• This is the oldest and until recently, the most
  prevalent form of computer
• Eg: most PCs, single CPU workstations and
  mainframes
Single Instruction, Multiple Data (SIMD):


• A type of parallel computer
• Single instruction: All processing units execute
  the same instruction at any given clock cycle
• Multiple data: Each processing unit can operate on
  a different data element
• This type of machine typically has an instruction
  dispatcher, a very high-bandwidth internal
  network, and a very large array of very small-
  capacity instruction units.
Single Instruction, Multiple Data
             (SIMD):
Single Instruction, Multiple Data (SIMD):

• Best suited for specialized problems characterized by a
  high degree of regularity,such as image processing.
• Synchronous (lockstep) and deterministic execution
• Two varieties: Processor Arrays and Vector Pipelines
• Examples (some extinct):
   o Processor Arrays: Connection Machine CM-2, Maspar MP-1,
     MP-2
   o Vector Pipelines: IBM 9000, Cray C90, Fujitsu VP, NEC SX-2,
     Hitachi S820
Multiple Instruction, Single Data
              (MISD):
• Few actual examples of this class of parallel
  computer have ever existed
• Some conceivable examples might be:
  – multiple frequency filters operating on a single
    signal stream
  – multiple cryptography algorithms attempting to
    crack a single coded message.
Multiple Instruction, Multiple Data
               (MIMD):
• Currently, the most common type of parallel computer
• Multiple Instruction: every processor may be executing a
  different instruction stream
• Multiple Data: every processor may be working with a
  different data stream
• Execution can be synchronous or asynchronous,
  deterministic or non- deterministic
• Examples: most current supercomputers, networked
  parallel computer "grids" and multi-processor SMP
  computers - including some types of PCs.
Multiple Instruction, Multiple Data
              (MIMD):
scalar

sequential            Lookahead

                               Functional
   I/E overlap                 parallelism

             Multiple
             Func. Units
                                     Pipeline

                    Implicit                 Explicit
                    Vector                   vector

                           Mem-to-Mem             Reg-to-Reg


                                    SIMD                MIMD

Assoc.                 Processor                Multicomputer   Multiprocessor
Processor              Array

More Related Content

What's hot

Unit 4 Real Time Operating System
Unit 4 Real Time Operating SystemUnit 4 Real Time Operating System
Unit 4 Real Time Operating System
Dr. Pankaj Zope
 
RISC (reduced instruction set computer)
RISC (reduced instruction set computer)RISC (reduced instruction set computer)
RISC (reduced instruction set computer)
LokmanArman
 
Micro Programmed Control Unit
Micro Programmed Control UnitMicro Programmed Control Unit
Micro Programmed Control Unit
Kamal Acharya
 
multiprocessors and multicomputers
 multiprocessors and multicomputers multiprocessors and multicomputers
multiprocessors and multicomputers
Pankaj Kumar Jain
 
ADDRESSING MODES
ADDRESSING MODESADDRESSING MODES
ADDRESSING MODES
Sadaf Rasheed
 
Pipelining and ILP (Instruction Level Parallelism)
Pipelining and ILP (Instruction Level Parallelism) Pipelining and ILP (Instruction Level Parallelism)
Pipelining and ILP (Instruction Level Parallelism)
A B Shinde
 
Parallel Algorithms
Parallel AlgorithmsParallel Algorithms
Parallel Algorithms
Dr Sandeep Kumar Poonia
 
Instruction format
Instruction formatInstruction format
Instruction format
chauhankapil
 
Operating System Scheduling Algorithms
Operating System Scheduling AlgorithmsOperating System Scheduling Algorithms
Operating System Scheduling Algorithms
International Institute of Information Technology (I²IT)
 
Hardware Multi-Threading
Hardware Multi-ThreadingHardware Multi-Threading
Hardware Multi-Threading
babuece
 
Parallel computing
Parallel computingParallel computing
Parallel computing
Vinay Gupta
 
pipelining
pipeliningpipelining
pipelining
Siddique Ibrahim
 
Pipeline hazard
Pipeline hazardPipeline hazard
Pipeline hazard
AJAL A J
 
Memory management
Memory managementMemory management
Memory management
Muhammad Fayyaz
 
Unit 5 Advanced Computer Architecture
Unit 5 Advanced Computer ArchitectureUnit 5 Advanced Computer Architecture
Unit 5 Advanced Computer Architecture
Balaji Vignesh
 
Direct access memory
Direct access memoryDirect access memory
Direct access memory
maliksiddique1
 
Parallel processing
Parallel processingParallel processing
Parallel processing
Syed Zaid Irshad
 
Data transfer and manipulation
Data transfer and manipulationData transfer and manipulation
Data transfer and manipulation
Sanjeev Patel
 
Memory organization (Computer architecture)
Memory organization (Computer architecture)Memory organization (Computer architecture)
Memory organization (Computer architecture)
Sandesh Jonchhe
 
Memory Organization
Memory OrganizationMemory Organization
Memory Organization
Acad
 

What's hot (20)

Unit 4 Real Time Operating System
Unit 4 Real Time Operating SystemUnit 4 Real Time Operating System
Unit 4 Real Time Operating System
 
RISC (reduced instruction set computer)
RISC (reduced instruction set computer)RISC (reduced instruction set computer)
RISC (reduced instruction set computer)
 
Micro Programmed Control Unit
Micro Programmed Control UnitMicro Programmed Control Unit
Micro Programmed Control Unit
 
multiprocessors and multicomputers
 multiprocessors and multicomputers multiprocessors and multicomputers
multiprocessors and multicomputers
 
ADDRESSING MODES
ADDRESSING MODESADDRESSING MODES
ADDRESSING MODES
 
Pipelining and ILP (Instruction Level Parallelism)
Pipelining and ILP (Instruction Level Parallelism) Pipelining and ILP (Instruction Level Parallelism)
Pipelining and ILP (Instruction Level Parallelism)
 
Parallel Algorithms
Parallel AlgorithmsParallel Algorithms
Parallel Algorithms
 
Instruction format
Instruction formatInstruction format
Instruction format
 
Operating System Scheduling Algorithms
Operating System Scheduling AlgorithmsOperating System Scheduling Algorithms
Operating System Scheduling Algorithms
 
Hardware Multi-Threading
Hardware Multi-ThreadingHardware Multi-Threading
Hardware Multi-Threading
 
Parallel computing
Parallel computingParallel computing
Parallel computing
 
pipelining
pipeliningpipelining
pipelining
 
Pipeline hazard
Pipeline hazardPipeline hazard
Pipeline hazard
 
Memory management
Memory managementMemory management
Memory management
 
Unit 5 Advanced Computer Architecture
Unit 5 Advanced Computer ArchitectureUnit 5 Advanced Computer Architecture
Unit 5 Advanced Computer Architecture
 
Direct access memory
Direct access memoryDirect access memory
Direct access memory
 
Parallel processing
Parallel processingParallel processing
Parallel processing
 
Data transfer and manipulation
Data transfer and manipulationData transfer and manipulation
Data transfer and manipulation
 
Memory organization (Computer architecture)
Memory organization (Computer architecture)Memory organization (Computer architecture)
Memory organization (Computer architecture)
 
Memory Organization
Memory OrganizationMemory Organization
Memory Organization
 

Similar to Lecture 2

Parallel & Distributed processing
Parallel & Distributed processingParallel & Distributed processing
Parallel & Distributed processing
Syed Zaid Irshad
 
Lec 2 (parallel design and programming)
Lec 2 (parallel design and programming)Lec 2 (parallel design and programming)
Lec 2 (parallel design and programming)
Sudarshan Mondal
 
unit 4.pptx
unit 4.pptxunit 4.pptx
unit 4.pptx
unit 4.pptxunit 4.pptx
Parallel architecture &programming
Parallel architecture &programmingParallel architecture &programming
Parallel architecture &programming
Ismail El Gayar
 
Parallel architecture-programming
Parallel architecture-programmingParallel architecture-programming
Parallel architecture-programming
Shaveta Banda
 
CSA unit5.pptx
CSA unit5.pptxCSA unit5.pptx
CSA unit5.pptx
AbcvDef
 
Week # 1.pdf
Week # 1.pdfWeek # 1.pdf
Week # 1.pdf
giddy5
 
Lecture 1 introduction to parallel and distributed computing
Lecture 1   introduction to parallel and distributed computingLecture 1   introduction to parallel and distributed computing
Lecture 1 introduction to parallel and distributed computing
Vajira Thambawita
 
Aca module 1
Aca module 1Aca module 1
Aca module 1
Avinash_N Rao
 
Multiprocessor.pptx
 Multiprocessor.pptx Multiprocessor.pptx
Multiprocessor.pptx
Muhammad54342
 
CA UNIT IV.pptx
CA UNIT IV.pptxCA UNIT IV.pptx
CA UNIT IV.pptx
ssuser9dbd7e
 
Computing notes
Computing notesComputing notes
Computing notes
thenraju24
 
Parallel Processors (SIMD)
Parallel Processors (SIMD) Parallel Processors (SIMD)
Parallel Processors (SIMD)
Ali Raza
 
Parallel Processors (SIMD)
Parallel Processors (SIMD) Parallel Processors (SIMD)
Parallel Processors (SIMD)
Ali Raza
 
Parallel computing persentation
Parallel computing persentationParallel computing persentation
Parallel computing persentation
VIKAS SINGH BHADOURIA
 
High performance computing
High performance computingHigh performance computing
High performance computing
punjab engineering college, chandigarh
 
intro, definitions, basic laws+.pptx
intro, definitions, basic laws+.pptxintro, definitions, basic laws+.pptx
intro, definitions, basic laws+.pptx
ssuser413a98
 
Array Processors & Architectural Classification Schemes_Computer Architecture...
Array Processors & Architectural Classification Schemes_Computer Architecture...Array Processors & Architectural Classification Schemes_Computer Architecture...
Array Processors & Architectural Classification Schemes_Computer Architecture...
Sumalatha A
 
Classification of Parallel Computers.pptx
Classification of Parallel Computers.pptxClassification of Parallel Computers.pptx
Classification of Parallel Computers.pptx
Neeraj Singh
 

Similar to Lecture 2 (20)

Parallel & Distributed processing
Parallel & Distributed processingParallel & Distributed processing
Parallel & Distributed processing
 
Lec 2 (parallel design and programming)
Lec 2 (parallel design and programming)Lec 2 (parallel design and programming)
Lec 2 (parallel design and programming)
 
unit 4.pptx
unit 4.pptxunit 4.pptx
unit 4.pptx
 
unit 4.pptx
unit 4.pptxunit 4.pptx
unit 4.pptx
 
Parallel architecture &programming
Parallel architecture &programmingParallel architecture &programming
Parallel architecture &programming
 
Parallel architecture-programming
Parallel architecture-programmingParallel architecture-programming
Parallel architecture-programming
 
CSA unit5.pptx
CSA unit5.pptxCSA unit5.pptx
CSA unit5.pptx
 
Week # 1.pdf
Week # 1.pdfWeek # 1.pdf
Week # 1.pdf
 
Lecture 1 introduction to parallel and distributed computing
Lecture 1   introduction to parallel and distributed computingLecture 1   introduction to parallel and distributed computing
Lecture 1 introduction to parallel and distributed computing
 
Aca module 1
Aca module 1Aca module 1
Aca module 1
 
Multiprocessor.pptx
 Multiprocessor.pptx Multiprocessor.pptx
Multiprocessor.pptx
 
CA UNIT IV.pptx
CA UNIT IV.pptxCA UNIT IV.pptx
CA UNIT IV.pptx
 
Computing notes
Computing notesComputing notes
Computing notes
 
Parallel Processors (SIMD)
Parallel Processors (SIMD) Parallel Processors (SIMD)
Parallel Processors (SIMD)
 
Parallel Processors (SIMD)
Parallel Processors (SIMD) Parallel Processors (SIMD)
Parallel Processors (SIMD)
 
Parallel computing persentation
Parallel computing persentationParallel computing persentation
Parallel computing persentation
 
High performance computing
High performance computingHigh performance computing
High performance computing
 
intro, definitions, basic laws+.pptx
intro, definitions, basic laws+.pptxintro, definitions, basic laws+.pptx
intro, definitions, basic laws+.pptx
 
Array Processors & Architectural Classification Schemes_Computer Architecture...
Array Processors & Architectural Classification Schemes_Computer Architecture...Array Processors & Architectural Classification Schemes_Computer Architecture...
Array Processors & Architectural Classification Schemes_Computer Architecture...
 
Classification of Parallel Computers.pptx
Classification of Parallel Computers.pptxClassification of Parallel Computers.pptx
Classification of Parallel Computers.pptx
 

More from Mr SMAK

Fyp list batch-2009 (project approval -rejected list)
Fyp list batch-2009 (project approval -rejected list)Fyp list batch-2009 (project approval -rejected list)
Fyp list batch-2009 (project approval -rejected list)
Mr SMAK
 
Assigments2009
Assigments2009Assigments2009
Assigments2009
Mr SMAK
 
Week1
Week1Week1
Week1
Mr SMAK
 
Evaluation of cellular network
Evaluation of cellular networkEvaluation of cellular network
Evaluation of cellular network
Mr SMAK
 
Common protocols
Common protocolsCommon protocols
Common protocols
Mr SMAK
 
Cellular network
Cellular networkCellular network
Cellular network
Mr SMAK
 
Lecture 6.1
Lecture  6.1Lecture  6.1
Lecture 6.1
Mr SMAK
 
Lecture 6
Lecture  6Lecture  6
Lecture 6
Mr SMAK
 
Parallel architecture
Parallel architectureParallel architecture
Parallel architecture
Mr SMAK
 
Lecture 3
Lecture 3Lecture 3
Lecture 3
Mr SMAK
 
Lecture 1
Lecture 1Lecture 1
Lecture 1
Mr SMAK
 
Lecture 6
Lecture  6Lecture  6
Lecture 6
Mr SMAK
 
Lecture 6.1
Lecture  6.1Lecture  6.1
Lecture 6.1
Mr SMAK
 
Chapter 2 ASE
Chapter 2 ASEChapter 2 ASE
Chapter 2 ASE
Mr SMAK
 
Structure of project plan and schedule
Structure of project plan and scheduleStructure of project plan and schedule
Structure of project plan and schedule
Mr SMAK
 
Proposal format
Proposal formatProposal format
Proposal format
Mr SMAK
 
Proposal announcement batch2009
Proposal announcement batch2009Proposal announcement batch2009
Proposal announcement batch2009
Mr SMAK
 
List ofsuparco projectsforuniversities
List ofsuparco projectsforuniversitiesList ofsuparco projectsforuniversities
List ofsuparco projectsforuniversities
Mr SMAK
 
Fyp timeline & assessment policy batch 2009
Fyp timeline & assessment policy batch 2009Fyp timeline & assessment policy batch 2009
Fyp timeline & assessment policy batch 2009
Mr SMAK
 
Fyp registration form batch 2009
Fyp registration form batch 2009Fyp registration form batch 2009
Fyp registration form batch 2009
Mr SMAK
 

More from Mr SMAK (20)

Fyp list batch-2009 (project approval -rejected list)
Fyp list batch-2009 (project approval -rejected list)Fyp list batch-2009 (project approval -rejected list)
Fyp list batch-2009 (project approval -rejected list)
 
Assigments2009
Assigments2009Assigments2009
Assigments2009
 
Week1
Week1Week1
Week1
 
Evaluation of cellular network
Evaluation of cellular networkEvaluation of cellular network
Evaluation of cellular network
 
Common protocols
Common protocolsCommon protocols
Common protocols
 
Cellular network
Cellular networkCellular network
Cellular network
 
Lecture 6.1
Lecture  6.1Lecture  6.1
Lecture 6.1
 
Lecture 6
Lecture  6Lecture  6
Lecture 6
 
Parallel architecture
Parallel architectureParallel architecture
Parallel architecture
 
Lecture 3
Lecture 3Lecture 3
Lecture 3
 
Lecture 1
Lecture 1Lecture 1
Lecture 1
 
Lecture 6
Lecture  6Lecture  6
Lecture 6
 
Lecture 6.1
Lecture  6.1Lecture  6.1
Lecture 6.1
 
Chapter 2 ASE
Chapter 2 ASEChapter 2 ASE
Chapter 2 ASE
 
Structure of project plan and schedule
Structure of project plan and scheduleStructure of project plan and schedule
Structure of project plan and schedule
 
Proposal format
Proposal formatProposal format
Proposal format
 
Proposal announcement batch2009
Proposal announcement batch2009Proposal announcement batch2009
Proposal announcement batch2009
 
List ofsuparco projectsforuniversities
List ofsuparco projectsforuniversitiesList ofsuparco projectsforuniversities
List ofsuparco projectsforuniversities
 
Fyp timeline & assessment policy batch 2009
Fyp timeline & assessment policy batch 2009Fyp timeline & assessment policy batch 2009
Fyp timeline & assessment policy batch 2009
 
Fyp registration form batch 2009
Fyp registration form batch 2009Fyp registration form batch 2009
Fyp registration form batch 2009
 

Recently uploaded

Chapter 5 - Managing Test Activities V4.0
Chapter 5 - Managing Test Activities V4.0Chapter 5 - Managing Test Activities V4.0
Chapter 5 - Managing Test Activities V4.0
Neeraj Kumar Singh
 
ScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking ReplicationScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking Replication
ScyllaDB
 
Building a Semantic Layer of your Data Platform
Building a Semantic Layer of your Data PlatformBuilding a Semantic Layer of your Data Platform
Building a Semantic Layer of your Data Platform
Enterprise Knowledge
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
Pablo Gómez Abajo
 
MongoDB to ScyllaDB: Technical Comparison and the Path to Success
MongoDB to ScyllaDB: Technical Comparison and the Path to SuccessMongoDB to ScyllaDB: Technical Comparison and the Path to Success
MongoDB to ScyllaDB: Technical Comparison and the Path to Success
ScyllaDB
 
Lee Barnes - Path to Becoming an Effective Test Automation Engineer.pdf
Lee Barnes - Path to Becoming an Effective Test Automation Engineer.pdfLee Barnes - Path to Becoming an Effective Test Automation Engineer.pdf
Lee Barnes - Path to Becoming an Effective Test Automation Engineer.pdf
leebarnesutopia
 
DynamoDB to ScyllaDB: Technical Comparison and the Path to Success
DynamoDB to ScyllaDB: Technical Comparison and the Path to SuccessDynamoDB to ScyllaDB: Technical Comparison and the Path to Success
DynamoDB to ScyllaDB: Technical Comparison and the Path to Success
ScyllaDB
 
Communications Mining Series - Zero to Hero - Session 2
Communications Mining Series - Zero to Hero - Session 2Communications Mining Series - Zero to Hero - Session 2
Communications Mining Series - Zero to Hero - Session 2
DianaGray10
 
intra-mart Accel series 2024 Spring updates_En
intra-mart Accel series 2024 Spring updates_Enintra-mart Accel series 2024 Spring updates_En
intra-mart Accel series 2024 Spring updates_En
NTTDATA INTRAMART
 
An Introduction to All Data Enterprise Integration
An Introduction to All Data Enterprise IntegrationAn Introduction to All Data Enterprise Integration
An Introduction to All Data Enterprise Integration
Safe Software
 
Containers & AI - Beauty and the Beast!?!
Containers & AI - Beauty and the Beast!?!Containers & AI - Beauty and the Beast!?!
Containers & AI - Beauty and the Beast!?!
Tobias Schneck
 
Call Girls Chennai ☎️ +91-7426014248 😍 Chennai Call Girl Beauty Girls Chennai...
Call Girls Chennai ☎️ +91-7426014248 😍 Chennai Call Girl Beauty Girls Chennai...Call Girls Chennai ☎️ +91-7426014248 😍 Chennai Call Girl Beauty Girls Chennai...
Call Girls Chennai ☎️ +91-7426014248 😍 Chennai Call Girl Beauty Girls Chennai...
anilsa9823
 
Call Girls Kochi 💯Call Us 🔝 7426014248 🔝 Independent Kochi Escorts Service Av...
Call Girls Kochi 💯Call Us 🔝 7426014248 🔝 Independent Kochi Escorts Service Av...Call Girls Kochi 💯Call Us 🔝 7426014248 🔝 Independent Kochi Escorts Service Av...
Call Girls Kochi 💯Call Us 🔝 7426014248 🔝 Independent Kochi Escorts Service Av...
dipikamodels1
 
Real-Time Persisted Events at Supercell
Real-Time Persisted Events at  SupercellReal-Time Persisted Events at  Supercell
Real-Time Persisted Events at Supercell
ScyllaDB
 
MongoDB vs ScyllaDB: Tractian’s Experience with Real-Time ML
MongoDB vs ScyllaDB: Tractian’s Experience with Real-Time MLMongoDB vs ScyllaDB: Tractian’s Experience with Real-Time ML
MongoDB vs ScyllaDB: Tractian’s Experience with Real-Time ML
ScyllaDB
 
APJC Introduction to ThousandEyes Webinar
APJC Introduction to ThousandEyes WebinarAPJC Introduction to ThousandEyes Webinar
APJC Introduction to ThousandEyes Webinar
ThousandEyes
 
Cyber Recovery Wargame
Cyber Recovery WargameCyber Recovery Wargame
Cyber Recovery Wargame
Databarracks
 
Automation Student Developers Session 3: Introduction to UI Automation
Automation Student Developers Session 3: Introduction to UI AutomationAutomation Student Developers Session 3: Introduction to UI Automation
Automation Student Developers Session 3: Introduction to UI Automation
UiPathCommunity
 
Elasticity vs. State? Exploring Kafka Streams Cassandra State Store
Elasticity vs. State? Exploring Kafka Streams Cassandra State StoreElasticity vs. State? Exploring Kafka Streams Cassandra State Store
Elasticity vs. State? Exploring Kafka Streams Cassandra State Store
ScyllaDB
 
Fuxnet [EN] .pdf
Fuxnet [EN]                                   .pdfFuxnet [EN]                                   .pdf
Fuxnet [EN] .pdf
Overkill Security
 

Recently uploaded (20)

Chapter 5 - Managing Test Activities V4.0
Chapter 5 - Managing Test Activities V4.0Chapter 5 - Managing Test Activities V4.0
Chapter 5 - Managing Test Activities V4.0
 
ScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking ReplicationScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking Replication
 
Building a Semantic Layer of your Data Platform
Building a Semantic Layer of your Data PlatformBuilding a Semantic Layer of your Data Platform
Building a Semantic Layer of your Data Platform
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
 
MongoDB to ScyllaDB: Technical Comparison and the Path to Success
MongoDB to ScyllaDB: Technical Comparison and the Path to SuccessMongoDB to ScyllaDB: Technical Comparison and the Path to Success
MongoDB to ScyllaDB: Technical Comparison and the Path to Success
 
Lee Barnes - Path to Becoming an Effective Test Automation Engineer.pdf
Lee Barnes - Path to Becoming an Effective Test Automation Engineer.pdfLee Barnes - Path to Becoming an Effective Test Automation Engineer.pdf
Lee Barnes - Path to Becoming an Effective Test Automation Engineer.pdf
 
DynamoDB to ScyllaDB: Technical Comparison and the Path to Success
DynamoDB to ScyllaDB: Technical Comparison and the Path to SuccessDynamoDB to ScyllaDB: Technical Comparison and the Path to Success
DynamoDB to ScyllaDB: Technical Comparison and the Path to Success
 
Communications Mining Series - Zero to Hero - Session 2
Communications Mining Series - Zero to Hero - Session 2Communications Mining Series - Zero to Hero - Session 2
Communications Mining Series - Zero to Hero - Session 2
 
intra-mart Accel series 2024 Spring updates_En
intra-mart Accel series 2024 Spring updates_Enintra-mart Accel series 2024 Spring updates_En
intra-mart Accel series 2024 Spring updates_En
 
An Introduction to All Data Enterprise Integration
An Introduction to All Data Enterprise IntegrationAn Introduction to All Data Enterprise Integration
An Introduction to All Data Enterprise Integration
 
Containers & AI - Beauty and the Beast!?!
Containers & AI - Beauty and the Beast!?!Containers & AI - Beauty and the Beast!?!
Containers & AI - Beauty and the Beast!?!
 
Call Girls Chennai ☎️ +91-7426014248 😍 Chennai Call Girl Beauty Girls Chennai...
Call Girls Chennai ☎️ +91-7426014248 😍 Chennai Call Girl Beauty Girls Chennai...Call Girls Chennai ☎️ +91-7426014248 😍 Chennai Call Girl Beauty Girls Chennai...
Call Girls Chennai ☎️ +91-7426014248 😍 Chennai Call Girl Beauty Girls Chennai...
 
Call Girls Kochi 💯Call Us 🔝 7426014248 🔝 Independent Kochi Escorts Service Av...
Call Girls Kochi 💯Call Us 🔝 7426014248 🔝 Independent Kochi Escorts Service Av...Call Girls Kochi 💯Call Us 🔝 7426014248 🔝 Independent Kochi Escorts Service Av...
Call Girls Kochi 💯Call Us 🔝 7426014248 🔝 Independent Kochi Escorts Service Av...
 
Real-Time Persisted Events at Supercell
Real-Time Persisted Events at  SupercellReal-Time Persisted Events at  Supercell
Real-Time Persisted Events at Supercell
 
MongoDB vs ScyllaDB: Tractian’s Experience with Real-Time ML
MongoDB vs ScyllaDB: Tractian’s Experience with Real-Time MLMongoDB vs ScyllaDB: Tractian’s Experience with Real-Time ML
MongoDB vs ScyllaDB: Tractian’s Experience with Real-Time ML
 
APJC Introduction to ThousandEyes Webinar
APJC Introduction to ThousandEyes WebinarAPJC Introduction to ThousandEyes Webinar
APJC Introduction to ThousandEyes Webinar
 
Cyber Recovery Wargame
Cyber Recovery WargameCyber Recovery Wargame
Cyber Recovery Wargame
 
Automation Student Developers Session 3: Introduction to UI Automation
Automation Student Developers Session 3: Introduction to UI AutomationAutomation Student Developers Session 3: Introduction to UI Automation
Automation Student Developers Session 3: Introduction to UI Automation
 
Elasticity vs. State? Exploring Kafka Streams Cassandra State Store
Elasticity vs. State? Exploring Kafka Streams Cassandra State StoreElasticity vs. State? Exploring Kafka Streams Cassandra State Store
Elasticity vs. State? Exploring Kafka Streams Cassandra State Store
 
Fuxnet [EN] .pdf
Fuxnet [EN]                                   .pdfFuxnet [EN]                                   .pdf
Fuxnet [EN] .pdf
 

Lecture 2

  • 2. Terminologies • Task – A logically discrete section of computational work. A task is typically a program or program-like set of instructions that is executed by a processor. • Parallel Task – A task that can be executed by multiple processors safely (yields correct results) • Serial Execution – Execution of a program sequentially, one statement at a time. In the simplest sense, this is what happens on a one processor machine. However, virtually all parallel tasks will have sections of a parallel program that must be executed serially. • Parallel Execution – Execution of a program by more than one task, with each task being able to execute the same or different statement at the same moment in time.
  • 3. Terminologies • Shared Memory – From a strictly hardware point of view, describes a computer architecture where all processors have direct (usually bus based) access to common physical memory. In a programming sense, it describes a model where parallel tasks all have the same "picture" of memory and can directly address and access the same logical memory locations regardless of where the physical memory actually exists. • Distributed Memory – In hardware, refers to network based memory access for physical memory that is not common. As a programming model, tasks can only logically "see" local machine memory and must use communications to access memory on other machines where other tasks are executing
  • 4. Terminologies • Communications – Parallel tasks typically need to exchange data. There are several ways this can be accomplished, such as through a shared memory bus or over a network, however the actual event of data exchange is commonly referred to as communications regardless of the method employed. • Synchronization – The coordination of parallel tasks in real time, very often associated with communications. Often implemented by establishing a synchronization point within an application where a task may not proceed further until another task(s) reaches the same or logically equivalent point. Synchronization usually involves waiting by at least one task, and can therefore cause a parallel application's wall clock execution time to increase
  • 5. Terminologies • Granularity – In parallel computing, granularity is a qualitative measure of the ratio of computation to communication. • Coarse: relatively large amounts of computational work are done between communication events • Fine: relatively small amounts of computational work are done between communication events • Observed Speedup – Observed speedup of a code which has been parallelized, defined as: wall-clock time of serial execution / wall-clock time of parallel execution – One of the simplest and most widely used indicators for
  • 6. von Neumann Architecture • For over 40 years, virtually all computers have followed a common machine model known as the von Neumann computer. Named after the Hungarian mathematician John von Neumann. • A von Neumann computer uses the stored- program concept. The CPU executes a stored program that specifies a sequence of read and write operations on the memory.
  • 7. von Neumann Architecture • Memory is used to store both program and data instructions • Program instructions are coded data which tell the computer to do something • Data is simply information to be used by the program • A central processing unit (CPU) gets instructions and/or data from memory, decodes the instructions and then sequentially performs them
  • 8. Flynn's Classical Taxonomy • There are different ways to classify parallel computers. One of the more widely used classifications, in use since 1966, is called Flynn's Taxonomy. • Flynn's taxonomy distinguishes multi-processor computer architectures according to how they can be classified along the two independent dimensions of Instruction and Data. Each of these dimensions can have only one of two possible states: Single or Multiple.
  • 9. Flynn's Classical Taxonomy • The matrix below defines the 4 possible classifications according to Flynn. •S I S D •S I M D •Single Instruction, Single Data •Single Instruction, Multiple Data •M I S D •M I M D •Multiple Instruction, Single •Multiple Instruction, Multiple Data Data
  • 10. Single Instruction, Single Data (SISD) • A serial (non-parallel) computer • Single instruction: only one instruction stream is being acted on by the CPU during any one clock cycle • Single data: only one data stream is being used as input during any one clock cycle • Deterministic execution • This is the oldest and until recently, the most prevalent form of computer • Eg: most PCs, single CPU workstations and mainframes
  • 11. Single Instruction, Multiple Data (SIMD): • A type of parallel computer • Single instruction: All processing units execute the same instruction at any given clock cycle • Multiple data: Each processing unit can operate on a different data element • This type of machine typically has an instruction dispatcher, a very high-bandwidth internal network, and a very large array of very small- capacity instruction units.
  • 13. Single Instruction, Multiple Data (SIMD): • Best suited for specialized problems characterized by a high degree of regularity,such as image processing. • Synchronous (lockstep) and deterministic execution • Two varieties: Processor Arrays and Vector Pipelines • Examples (some extinct): o Processor Arrays: Connection Machine CM-2, Maspar MP-1, MP-2 o Vector Pipelines: IBM 9000, Cray C90, Fujitsu VP, NEC SX-2, Hitachi S820
  • 14. Multiple Instruction, Single Data (MISD): • Few actual examples of this class of parallel computer have ever existed • Some conceivable examples might be: – multiple frequency filters operating on a single signal stream – multiple cryptography algorithms attempting to crack a single coded message.
  • 15. Multiple Instruction, Multiple Data (MIMD): • Currently, the most common type of parallel computer • Multiple Instruction: every processor may be executing a different instruction stream • Multiple Data: every processor may be working with a different data stream • Execution can be synchronous or asynchronous, deterministic or non- deterministic • Examples: most current supercomputers, networked parallel computer "grids" and multi-processor SMP computers - including some types of PCs.
  • 17.
  • 18. scalar sequential Lookahead Functional I/E overlap parallelism Multiple Func. Units Pipeline Implicit Explicit Vector vector Mem-to-Mem Reg-to-Reg SIMD MIMD Assoc. Processor Multicomputer Multiprocessor Processor Array
  翻译: