A distributed system is a network that consists of autonomous computers that are connected using a distribution middleware. They help in sharing different resources and capabilities to provide users with a single and integrated coherent network.
The document discusses the divide and conquer algorithm design technique. It begins by explaining the basic approach of divide and conquer which is to (1) divide the problem into subproblems, (2) conquer the subproblems by solving them recursively, and (3) combine the solutions to the subproblems into a solution for the original problem. It then provides merge sort as a specific example of a divide and conquer algorithm for sorting a sequence. It explains that merge sort divides the sequence in half recursively until individual elements remain, then combines the sorted halves back together to produce the fully sorted sequence.
Strassen's algorithm improves on the basic matrix multiplication algorithm which runs in O(N3) time. It achieves this by dividing the matrices into sub-matrices and performing 7 multiplications and 18 additions on the sub-matrices, rather than the 8 multiplications of the basic algorithm. This results in a runtime of O(N2.81) using divide and conquer, providing an asymptotic improvement over the basic O(N3) algorithm.
Introduction to Matrix Chain Multiplication algorithm with an example. Matrix Chain Products algorithm comes under Dynamic Programming concept. Done for the course Advanced Data Structures and Algorithms.
this is a briefer overview about the Big O Notation. Big O Notaion are useful to check the Effeciency of an algorithm and to check its limitation at higher value. with big o notation some examples are also shown about its cases and some functions in c++ are also described.
This document summarizes basic communication operations for parallel computing including:
- One-to-all broadcast and all-to-one reduction which involve sending a message from one processor to all others or combining messages from all processors to one.
- All-to-all broadcast and reduction where all processors simultaneously broadcast or reduce messages.
- Collective operations like all-reduce and prefix-sum which combine messages from all processors using associative operators.
- Examples of implementing these operations on different network topologies like rings, meshes and hypercubes are presented along with analyzing their communication costs. The document provides an overview of fundamental communication patterns in parallel computing.
The document discusses inference in first-order logic. It provides a brief history of reasoning and logic. It then discusses reducing first-order inference to propositional inference using techniques like universal instantiation and existential instantiation. It introduces the concepts of unification and generalized modus ponens to perform inference in first-order logic. Forward chaining and resolution are also discussed as algorithms for performing inference in first-order logic.
The document discusses directed acyclic graphs (DAGs) and how they can be used to represent basic blocks of code. It describes how a DAG is constructed from three-address statements, with nodes labeled by variables, operators, or unique identifiers. Interior nodes represent computed values and leaves represent variables or constants. The DAG construction process creates nodes and links them based on the statements. DAGs are useful for detecting common subexpressions, determining which variables are used in a block, and which statements compute values used outside the block. Array accesses, pointers, and procedure calls require additional rules when constructing DAGs to properly capture dependencies.
- Recurrences describe functions in terms of their values on smaller inputs and arise when algorithms contain recursive calls to themselves.
- To analyze the running time of recursive algorithms, the recurrence must be solved to find an explicit formula or bound the expression in terms of n.
- Examples of recurrences and their solutions are given, including binary search (O(log n)), dividing the input in half at each step (O(n)), and dividing the input in half but examining all items (O(n)).
- Methods for solving recurrences include iteration, substitution, and using recursion trees to "guess" the solution.
The document discusses the divide and conquer algorithm design technique. It begins by explaining the basic approach of divide and conquer which is to (1) divide the problem into subproblems, (2) conquer the subproblems by solving them recursively, and (3) combine the solutions to the subproblems into a solution for the original problem. It then provides merge sort as a specific example of a divide and conquer algorithm for sorting a sequence. It explains that merge sort divides the sequence in half recursively until individual elements remain, then combines the sorted halves back together to produce the fully sorted sequence.
Strassen's algorithm improves on the basic matrix multiplication algorithm which runs in O(N3) time. It achieves this by dividing the matrices into sub-matrices and performing 7 multiplications and 18 additions on the sub-matrices, rather than the 8 multiplications of the basic algorithm. This results in a runtime of O(N2.81) using divide and conquer, providing an asymptotic improvement over the basic O(N3) algorithm.
Introduction to Matrix Chain Multiplication algorithm with an example. Matrix Chain Products algorithm comes under Dynamic Programming concept. Done for the course Advanced Data Structures and Algorithms.
this is a briefer overview about the Big O Notation. Big O Notaion are useful to check the Effeciency of an algorithm and to check its limitation at higher value. with big o notation some examples are also shown about its cases and some functions in c++ are also described.
This document summarizes basic communication operations for parallel computing including:
- One-to-all broadcast and all-to-one reduction which involve sending a message from one processor to all others or combining messages from all processors to one.
- All-to-all broadcast and reduction where all processors simultaneously broadcast or reduce messages.
- Collective operations like all-reduce and prefix-sum which combine messages from all processors using associative operators.
- Examples of implementing these operations on different network topologies like rings, meshes and hypercubes are presented along with analyzing their communication costs. The document provides an overview of fundamental communication patterns in parallel computing.
The document discusses inference in first-order logic. It provides a brief history of reasoning and logic. It then discusses reducing first-order inference to propositional inference using techniques like universal instantiation and existential instantiation. It introduces the concepts of unification and generalized modus ponens to perform inference in first-order logic. Forward chaining and resolution are also discussed as algorithms for performing inference in first-order logic.
The document discusses directed acyclic graphs (DAGs) and how they can be used to represent basic blocks of code. It describes how a DAG is constructed from three-address statements, with nodes labeled by variables, operators, or unique identifiers. Interior nodes represent computed values and leaves represent variables or constants. The DAG construction process creates nodes and links them based on the statements. DAGs are useful for detecting common subexpressions, determining which variables are used in a block, and which statements compute values used outside the block. Array accesses, pointers, and procedure calls require additional rules when constructing DAGs to properly capture dependencies.
- Recurrences describe functions in terms of their values on smaller inputs and arise when algorithms contain recursive calls to themselves.
- To analyze the running time of recursive algorithms, the recurrence must be solved to find an explicit formula or bound the expression in terms of n.
- Examples of recurrences and their solutions are given, including binary search (O(log n)), dividing the input in half at each step (O(n)), and dividing the input in half but examining all items (O(n)).
- Methods for solving recurrences include iteration, substitution, and using recursion trees to "guess" the solution.
Graph mining involves discovering frequent subgraphs, patterns, or substructures from a graph database. It has applications in domains like cheminformatics, bioinformatics, social network analysis, and knowledge discovery. There are two main approaches for frequent subgraph mining - Apriori-based approaches that generate candidates level-wise and pattern growth approaches that extend frequent subgraphs. The gSpan algorithm reduces redundant searching by using a depth-first search ordering of the graphs. Mining closed, maximal or dense frequent subgraphs can further reduce the number of patterns discovered. Applications include graph indexing, substructure similarity search, and graph classification or clustering.
In all-reduce, each node starts with a buffer of size m and the final results of the operation are identical buffers of size m on each node that are formed by combining the original p buffers using an associative operator.
This document provides an overview of symmetric encryption techniques, including:
- Symmetric encryption uses a shared secret key for both encryption and decryption.
- Classical encryption algorithms like the Caesar cipher and monoalphabetic substitution ciphers can be broken through frequency analysis.
- The Playfair cipher improved security over monoalphabetic ciphers by encrypting digrams, but it can still be broken with enough ciphertext.
- Polyalphabetic ciphers like the Vigenère cipher further increase security by using multiple cipher alphabets selected by a keyword.
This document discusses the merge sort algorithm for sorting a sequence of numbers. It begins by introducing the divide and conquer approach, which merge sort uses. It then provides an example of how merge sort works, dividing the sequence into halves, sorting the halves recursively, and then merging the sorted halves together. The document proceeds to provide pseudocode for the merge sort and merge algorithms. It analyzes the running time of merge sort using recursion trees, determining that it runs in O(n log n) time. Finally, it covers techniques for solving recurrence relations that arise in algorithms like divide and conquer approaches.
Association Rule Learning Part 1: Frequent Itemset GenerationKnoldus Inc.
A methodology useful for discovering interesting relationships hidden in large data sets. The uncovered relationships can be presented in the form of association rules.
The document discusses heap data structures and their use in priority queues and heapsort. It defines a heap as a complete binary tree stored in an array. Each node stores a value, with the heap property being that a node's value is greater than or equal to its children's values (for a max heap). Algorithms like Max-Heapify, Build-Max-Heap, Heap-Extract-Max, and Heap-Increase-Key are presented to maintain the heap property during operations. Priority queues use heaps to efficiently retrieve the maximum element, while heapsort sorts an array by building a max heap and repeatedly extracting elements.
Bayesian linear regression models the relationship between variables by characterizing the mean of one parameter as a weighted sum of other variables. It determines prior distributions for the regressors and finds the posterior distribution, which is the likelihood function multiplied by the prior, normalized. This allows the model to produce probabilistic outputs and account for uncertainty. Bayesian linear regression is useful when datasets are small but becomes less efficient than other approaches as data size increases.
This document provides an overview of first-order logic in artificial intelligence:
- First-order logic extends propositional logic by adding objects, relations, and functions to represent knowledge. Objects can include people and numbers, while relations include concepts like "brother of" and functions like "father of".
- A sentence in first-order logic contains a predicate and a subject, represented by a variable. For example, "tall(John)" asserts that John is tall. Quantifiers like "forall" and "exists" are used to structure sentences.
- First-order logic contains constants, variables, predicates, functions, connectives, equality, and quantifiers as its basic elements.
The document discusses classical AI planning and different planning approaches. It introduces state-space planning which searches for a sequence of state transformations, and plan-space planning which searches for a plan satisfying certain conditions. It also discusses hierarchical planning which decomposes tasks into simpler subtasks, and universal classical planning which uses different refinement techniques including state-space and plan-space refinements. Classical planning makes simplifying assumptions but its principles can still be applied to games with some workarounds.
k-means clustering aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean, serving as a prototype of the cluster. This results in a partitioning of the data space into Voronoi cells.
Quick sort Algorithm Discussion And AnalysisSNJ Chaudhary
Quicksort is a divide-and-conquer algorithm that works by partitioning an array around a pivot element and recursively sorting the subarrays. In the average case, it has an efficiency of Θ(n log n) time as the partitioning typically divides the array into balanced subproblems. However, in the worst case of an already sorted array, it can be Θ(n^2) time due to highly unbalanced partitioning. Randomizing the choice of pivot helps avoid worst-case scenarios and achieve average-case efficiency in practice, making quicksort very efficient and commonly used.
This document discusses code-tuning techniques to improve performance without major refactoring. It describes techniques for optimizing logic, loops, data transformations, expressions, routines, and recoding performance-critical sections in assembler. Specific techniques covered include short-circuit evaluation, ordering tests by frequency, substituting table lookups, unswitching loops, jamming loops, loop unrolling, minimizing work inside loops, and rewriting routines inline. The goal is to make small, targeted changes to improve efficiency rather than large-scale redesigns.
The document discusses the Apriori algorithm for finding frequent itemsets in transactional data. It begins by defining key concepts like itemsets, support count, and frequent itemsets. It then explains the core steps of the Apriori algorithm: generating candidate itemsets from frequent k-itemsets, scanning the database to determine frequent (k+1)-itemsets, and pruning infrequent supersets. The document also introduces optimizations like the AprioriTid algorithm, which makes a single pass over the data using data structures to count support.
This document provides an introduction to association rule mining. It begins with an overview of association rule mining and its application to market basket analysis. It then discusses key concepts like support, confidence and interestingness of rules. The document introduces the Apriori algorithm for mining association rules, which works in two steps: 1) generating frequent itemsets and 2) generating rules from frequent itemsets. It provides examples of how Apriori works and discusses challenges in association rule mining like multiple database scans and candidate generation.
This document provides an introduction to the Master Theorem, which can be used to determine the asymptotic runtime of recursive algorithms. It presents the three main conditions of the Master Theorem and examples of applying it to solve recurrence relations. It also notes some pitfalls in using the Master Theorem and briefly introduces a fourth condition for cases where the non-recursive term is polylogarithmic rather than polynomial.
This document discusses graph mining, including its motivation, applications, and algorithms. Graph mining aims to discover repetitive subgraphs in graph datasets. It has many applications including analyzing chemical compounds, biological networks, program flows, and social networks. The document outlines several graph mining algorithms, including the Apriori-based FSG algorithm, the DFS-based gSpan algorithm, and the greedy Subdue algorithm. It also distinguishes between the transaction setting and single graph setting for graph mining problems.
This document discusses various synchronization issues in distributed systems including clock synchronization, event ordering, mutual exclusion, and deadlock. It describes how computer clocks are implemented and different clock synchronization algorithms like centralized and distributed algorithms. It explains logical clocks and happened-before relation for event ordering. Different approaches for mutual exclusion like centralized, distributed, and token-passing are outlined. The four conditions for deadlock and different strategies like avoidance, prevention, and detection and recovery are summarized. Resource allocation graphs and wait-for graphs are introduced for modeling deadlocks.
Logical clocks are mechanisms for capturing chronological and causal relationships in distributed systems. Lamport introduced logical clocks using timestamps assigned to events to define a "happens before" relation between events. Vector clocks extend logical timestamps to capture causality more accurately by maintaining a vector of timestamps, with one entry per process. Matrix clocks further extend this idea by maintaining a matrix to represent processes' knowledge of other processes' logical clocks.
Graph mining involves discovering frequent subgraphs, patterns, or substructures from a graph database. It has applications in domains like cheminformatics, bioinformatics, social network analysis, and knowledge discovery. There are two main approaches for frequent subgraph mining - Apriori-based approaches that generate candidates level-wise and pattern growth approaches that extend frequent subgraphs. The gSpan algorithm reduces redundant searching by using a depth-first search ordering of the graphs. Mining closed, maximal or dense frequent subgraphs can further reduce the number of patterns discovered. Applications include graph indexing, substructure similarity search, and graph classification or clustering.
In all-reduce, each node starts with a buffer of size m and the final results of the operation are identical buffers of size m on each node that are formed by combining the original p buffers using an associative operator.
This document provides an overview of symmetric encryption techniques, including:
- Symmetric encryption uses a shared secret key for both encryption and decryption.
- Classical encryption algorithms like the Caesar cipher and monoalphabetic substitution ciphers can be broken through frequency analysis.
- The Playfair cipher improved security over monoalphabetic ciphers by encrypting digrams, but it can still be broken with enough ciphertext.
- Polyalphabetic ciphers like the Vigenère cipher further increase security by using multiple cipher alphabets selected by a keyword.
This document discusses the merge sort algorithm for sorting a sequence of numbers. It begins by introducing the divide and conquer approach, which merge sort uses. It then provides an example of how merge sort works, dividing the sequence into halves, sorting the halves recursively, and then merging the sorted halves together. The document proceeds to provide pseudocode for the merge sort and merge algorithms. It analyzes the running time of merge sort using recursion trees, determining that it runs in O(n log n) time. Finally, it covers techniques for solving recurrence relations that arise in algorithms like divide and conquer approaches.
Association Rule Learning Part 1: Frequent Itemset GenerationKnoldus Inc.
A methodology useful for discovering interesting relationships hidden in large data sets. The uncovered relationships can be presented in the form of association rules.
The document discusses heap data structures and their use in priority queues and heapsort. It defines a heap as a complete binary tree stored in an array. Each node stores a value, with the heap property being that a node's value is greater than or equal to its children's values (for a max heap). Algorithms like Max-Heapify, Build-Max-Heap, Heap-Extract-Max, and Heap-Increase-Key are presented to maintain the heap property during operations. Priority queues use heaps to efficiently retrieve the maximum element, while heapsort sorts an array by building a max heap and repeatedly extracting elements.
Bayesian linear regression models the relationship between variables by characterizing the mean of one parameter as a weighted sum of other variables. It determines prior distributions for the regressors and finds the posterior distribution, which is the likelihood function multiplied by the prior, normalized. This allows the model to produce probabilistic outputs and account for uncertainty. Bayesian linear regression is useful when datasets are small but becomes less efficient than other approaches as data size increases.
This document provides an overview of first-order logic in artificial intelligence:
- First-order logic extends propositional logic by adding objects, relations, and functions to represent knowledge. Objects can include people and numbers, while relations include concepts like "brother of" and functions like "father of".
- A sentence in first-order logic contains a predicate and a subject, represented by a variable. For example, "tall(John)" asserts that John is tall. Quantifiers like "forall" and "exists" are used to structure sentences.
- First-order logic contains constants, variables, predicates, functions, connectives, equality, and quantifiers as its basic elements.
The document discusses classical AI planning and different planning approaches. It introduces state-space planning which searches for a sequence of state transformations, and plan-space planning which searches for a plan satisfying certain conditions. It also discusses hierarchical planning which decomposes tasks into simpler subtasks, and universal classical planning which uses different refinement techniques including state-space and plan-space refinements. Classical planning makes simplifying assumptions but its principles can still be applied to games with some workarounds.
k-means clustering aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean, serving as a prototype of the cluster. This results in a partitioning of the data space into Voronoi cells.
Quick sort Algorithm Discussion And AnalysisSNJ Chaudhary
Quicksort is a divide-and-conquer algorithm that works by partitioning an array around a pivot element and recursively sorting the subarrays. In the average case, it has an efficiency of Θ(n log n) time as the partitioning typically divides the array into balanced subproblems. However, in the worst case of an already sorted array, it can be Θ(n^2) time due to highly unbalanced partitioning. Randomizing the choice of pivot helps avoid worst-case scenarios and achieve average-case efficiency in practice, making quicksort very efficient and commonly used.
This document discusses code-tuning techniques to improve performance without major refactoring. It describes techniques for optimizing logic, loops, data transformations, expressions, routines, and recoding performance-critical sections in assembler. Specific techniques covered include short-circuit evaluation, ordering tests by frequency, substituting table lookups, unswitching loops, jamming loops, loop unrolling, minimizing work inside loops, and rewriting routines inline. The goal is to make small, targeted changes to improve efficiency rather than large-scale redesigns.
The document discusses the Apriori algorithm for finding frequent itemsets in transactional data. It begins by defining key concepts like itemsets, support count, and frequent itemsets. It then explains the core steps of the Apriori algorithm: generating candidate itemsets from frequent k-itemsets, scanning the database to determine frequent (k+1)-itemsets, and pruning infrequent supersets. The document also introduces optimizations like the AprioriTid algorithm, which makes a single pass over the data using data structures to count support.
This document provides an introduction to association rule mining. It begins with an overview of association rule mining and its application to market basket analysis. It then discusses key concepts like support, confidence and interestingness of rules. The document introduces the Apriori algorithm for mining association rules, which works in two steps: 1) generating frequent itemsets and 2) generating rules from frequent itemsets. It provides examples of how Apriori works and discusses challenges in association rule mining like multiple database scans and candidate generation.
This document provides an introduction to the Master Theorem, which can be used to determine the asymptotic runtime of recursive algorithms. It presents the three main conditions of the Master Theorem and examples of applying it to solve recurrence relations. It also notes some pitfalls in using the Master Theorem and briefly introduces a fourth condition for cases where the non-recursive term is polylogarithmic rather than polynomial.
This document discusses graph mining, including its motivation, applications, and algorithms. Graph mining aims to discover repetitive subgraphs in graph datasets. It has many applications including analyzing chemical compounds, biological networks, program flows, and social networks. The document outlines several graph mining algorithms, including the Apriori-based FSG algorithm, the DFS-based gSpan algorithm, and the greedy Subdue algorithm. It also distinguishes between the transaction setting and single graph setting for graph mining problems.
This document discusses various synchronization issues in distributed systems including clock synchronization, event ordering, mutual exclusion, and deadlock. It describes how computer clocks are implemented and different clock synchronization algorithms like centralized and distributed algorithms. It explains logical clocks and happened-before relation for event ordering. Different approaches for mutual exclusion like centralized, distributed, and token-passing are outlined. The four conditions for deadlock and different strategies like avoidance, prevention, and detection and recovery are summarized. Resource allocation graphs and wait-for graphs are introduced for modeling deadlocks.
Logical clocks are mechanisms for capturing chronological and causal relationships in distributed systems. Lamport introduced logical clocks using timestamps assigned to events to define a "happens before" relation between events. Vector clocks extend logical timestamps to capture causality more accurately by maintaining a vector of timestamps, with one entry per process. Matrix clocks further extend this idea by maintaining a matrix to represent processes' knowledge of other processes' logical clocks.
This document discusses distributed mutual exclusion algorithms. It describes Lamport's algorithm which uses logical timestamps and message passing to ensure only one process is in the critical section at a time. It also covers Ricart-Agarwala's improvement which reduces the number of messages. Maekawa's algorithm is discussed which obtains permission from a quorum or subset of processes. Finally, it summarizes the Suzuki-Kasami and Raymond token-based algorithms.
Chapter 5 discusses synchronization in distributed systems. Synchronization mechanisms are needed to enforce correct interaction between processes that share resources and run concurrently. Clock synchronization and event ordering are important synchronization techniques. Clock synchronization aims to keep clocks across distributed nodes close together despite unpredictable delays. It can be achieved through centralized or distributed algorithms. Event ordering ensures a total order of all events in a distributed system through happened-before relations and logical clocks.
This document summarizes key concepts around synchronization in distributed systems including clock synchronization, logical clocks, election algorithms, and mutual exclusion. It discusses how physical clocks become unsynchronized over time, introduces Lamport timestamps and logical clocks, describes the Bully and Ring election algorithms, and covers a centralized mutual exclusion algorithm that uses a coordinator to grant processes permission to enter a critical region.
Distributed Systems (3rd Edition) Chapter 06 discusses coordination through clock synchronization and logical clocks. It describes several clock synchronization algorithms like Network Time Protocol that aim to keep physical clocks precise and accurate. Logical clocks like Lamport's logical clocks attach timestamps to events in a way that preserves causality without a global clock. Vector clocks were developed to guarantee causal precedence between timestamps. Total-ordered multicast uses logical clocks to ensure all processes see messages in the same order.
This document discusses various techniques for process synchronization including the critical section problem, semaphores, and classical synchronization problems like the bounded buffer, readers-writers, and dining philosophers problems. It provides code examples to illustrate how semaphores can be used to synchronize access to shared resources and ensure mutual exclusion between concurrent processes.
This document discusses various techniques for process synchronization including the critical section problem, semaphores, and classical synchronization problems like the bounded buffer, readers-writers, and dining philosophers problems. It provides code examples to illustrate how semaphores can be used to synchronize access to shared resources and ensure mutual exclusion between concurrent processes.
Ch-2 final exam documet compler design elementsMAHERMOHAMED27
The "Project Risk Management" course transformed me from a passive observer of risk to a proactive risk management champion. Here are some key learnings that will forever change my approach to projects:
The Proactive Mindset: I transitioned from simply reacting to problems to anticipating and mitigating them. The course emphasized the importance of proactive risk identification through techniques like brainstorming, SWOT analysis, and FMEA (Failure Mode and Effect Analysis). This allows for early intervention and prevents minor issues from snowballing into major roadblocks.
Risk Assessment and Prioritization: I learned to assess the likelihood and impact of each identified risk. The course introduced qualitative and quantitative risk analysis methods, allowing me to prioritize risks based on their potential severity. This empowers me to focus resources on the most critical threats to project success.
Developing Response Strategies: The course equipped me with a toolbox of risk response strategies. I learned about risk avoidance, mitigation, transference, and acceptance strategies, allowing me to choose the most appropriate approach for each risk. For example, I can now advocate for additional training to mitigate a knowledge gap risk or build buffer time into the schedule to address potential delays.
Communication and Monitoring: The course highlighted the importance of clear communication regarding risks. I learned to effectively communicate risks to stakeholders, ensuring everyone is aware of potential challenges and mitigation plans. Additionally, I gained valuable insights into risk monitoring and tracking, allowing for continuous evaluation and adaptation as the project progresses.
In essence, "Project Risk Management" equipped me with the knowledge and tools to navigate the inevitable uncertainties of projects. By embracing a proactive approach, I can now lead projects with greater confidence, increasing the chances of achieving successful outcomes.
This document discusses algorithm analysis and complexity. It introduces algorithm analysis as a way to predict and compare algorithm performance. Different algorithms for computing factorials and finding the maximum subsequence sum are presented, along with their time complexities. The importance of efficient algorithms for problems involving large datasets is discussed.
The document discusses various code tuning techniques to optimize code performance, including unswitching loops, unrolling loops, caching frequently used values, initializing data at compile time, using sentinels in search loops, putting the busiest loop on the inside, pre-computing results, and using integers instead of floating point variables. It provides examples of each technique in different programming languages and the measured time savings from applying each optimization, with some optimizations yielding up to 90% faster execution time. The key message is that minor changes to code implementation through these techniques can significantly improve performance, but optimizations must be carefully tested.
ODSC 2019: Sessionisation via stochastic periods for root event identificationKuldeep Jiwani
In todays world majority of information is generated by self sustaining systems like various kinds of bots, crawlers, servers, various online services, etc. This information is flowing on the axis of time and is generated by these actors under some complex logic. For example, a stream of buy/sell order requests by an Order Gateway in financial world, or a stream of web requests by a monitoring / crawling service in the web world, or may be a hacker's bot sitting on internet and attacking various computers. Although we may not be able to know the motive or intention behind these data sources. But via some unsupervised techniques we can try to infer the pattern or correlate the events based on their multiple occurrences on the axis of time. Associating a chain of events in order of time helps in doing a root event analysis. In certain cases a time ordered correlation and root event identification is good enough to automatically identify signatures of various malicious actors and take appropriate corrective actions to stop cyber attacks, stop malicious social campaigns, etc.
Sessionisation is one such unsupervised technique that tries to find the signal in a stream of events associated with a timestamp. In the ideal world it would resolve to finding periods with a mixture of sinusoidal waves. But for the real world this is a much complex activity, as even the systematic events generated by machines over the internet behave in a much erratic manner. So the notion of a period for a signal also changes in the real world. We can no longer associate it with a number, it has to be treated as a random variable, with expected values and associated variance. Hence we need to model "Stochastic periods" and learn their probability distributions in an unsupervised manner.
The main focus of this talk will be to showcase applied data science techniques to discover stochastic periods. There are many ways to obtain periods in data, so the journey would begin by a walk through of existing techniques like FFT (Fast Fourier Transform) then discuss about Gaussian Mixture Models. After highlighting the short comings of these techniques we will succinctly explain one of the most general non-parametric Bayesian approaches to solve this problem. Without going too deep in the complex math, we will get back to applied data science and discuss a much simpler technique that can solve the same problem if certain assumptions are satisfied.
In this talk we will demonstrate some time based pattern we discovered while working on a security analytics use case that uses Sessionisation. In the talk we will demonstrate such patterns based on an open source malware attack datasets that is available publicly.
Key concepts explained in talk: Sessionisation, Bayesian techniques of Machine Learning, Gaussian Mixture Models, Kernel density estimation, FFT, stochastic periods, probabilistic modelling, Bayesian non-parametric methods
The document discusses correctness of algorithms and time complexity analysis. It covers using loop invariants and mathematical induction to prove correctness. Loop invariants must be true before and after each loop iteration. They help reason about algorithm behavior and establish correctness. The document provides an example of designing a max finding algorithm and establishing an appropriate loop invariant. It also briefly discusses using class invariants in object-oriented programming.
The document discusses process synchronization and concurrency control techniques in operating systems, including classical problems like the bounded buffer problem, readers-writers problem, and dining philosophers problem. It covers synchronization primitives like semaphores, mutexes, and monitors that allow processes to synchronize access to shared resources and coordinate their activities to prevent race conditions and deadlocks. The key challenges of mutual exclusion, progress, and bounded waiting in solving the critical section problem are also summarized.
1) Logical clocks are used to provide a total ordering of events in an asynchronous distributed system where there is no global clock. Each process maintains a local logical clock that is incremented for compute and send events. The clock is updated based on timestamp values piggybacked in received messages.
2) The happened-before relation defines a partial ordering of events based on causality. Logical clocks extend this to a total ordering but the ordering is arbitrary and may not match the perceived order of users.
3) While logical clocks provide total ordering, the timestamps do not preclude concurrent events from having the same value. Vector clocks were developed to address this limitation of Lamport clocks.
Vadym Khondar is a senior software engineer with 8 years of experience, including 2.5 years at EPAM. He leads a development team that works on web and JavaScript projects. The document discusses reactive programming, including its benefits of responsiveness, resilience, and other qualities. Examples demonstrate using streams, behaviors, and other reactive concepts to write more declarative and asynchronous code.
This document discusses processes and their interaction in operating systems. It defines a process as a program in execution that includes code, data, threads of execution, and resources. It describes how operating systems virtualize CPUs and memory to allow multiple processes to run concurrently. It discusses different types of process interactions including competition over shared resources, which can cause race conditions, and cooperation through synchronization and message passing. It presents several algorithms to solve the critical section problem of mutual exclusion when processes access shared resources.
MODELING OF DISTRIBUTED MUTUAL EXCLUSION SYSTEM USING EVENT-B cscpconf
The problem of mutual exclusion arises in distributed systems whenever shared resources are concurrently accessed by several sites. For correctness, it is required that shared resource must be accessed by a single site at a time. To decide, which site execute the critical section next, each site communicate with a set of other sites. A systematic approach is essential to formulate an accurate speciation. Formal methods are mathematical techniques that provide systematic approach for building and verification of model. We have used Event-B as a formal technique for construction of our model. Event-B is event driven approach which
is used to develop formal models of distributed systems .It supports generation and discharge of proof obligations arising due to consistency checking. In this paper, we outline a formal construction of model of Lamport's mutual exclusion algorithm for distributed system using Event-B. We have considered vector clock instead of using Lam-port's scalar clock for the purpose of message's time stamping
Modeling of distributed mutual exclusion system using event bcsandit
The problem of mutual exclusion arises in distributed systems whenever shared resources are concurrently
accessed by several sites. For correctness, it is required that shared resource must be accessed by a single
site at a time. To decide, which site execute the critical section next, each site communicate with a set of
other sites. A systematic approach is essential to formulate an accurate speciation. Formal methods are
mathematical techniques that provide systematic approach for building and verification of model. We have
used Event-B as a formal technique for construction of our model. Event-B is event driven approach which
is used to develop formal models of distributed systems .It supports generation and discharge of proof
obligations arising due to consistency checking. In this paper, we outline a formal construction of model of
Lamport's mutual exclusion algorithm for distributed system using Event-B. We have considered vector
clock instead of using Lam-port's scalar clock for the purpose of message's time stamping.
This is an overview of my current metallic design and engineering knowledge base built up over my professional career and two MSc degrees : - MSc in Advanced Manufacturing Technology University of Portsmouth graduated 1st May 1998, and MSc in Aircraft Engineering Cranfield University graduated 8th June 2007.
Covid Management System Project Report.pdfKamal Acharya
CoVID-19 sprang up in Wuhan China in November 2019 and was declared a pandemic by the in January 2020 World Health Organization (WHO). Like the Spanish flu of 1918 that claimed millions of lives, the COVID-19 has caused the demise of thousands with China, Italy, Spain, USA and India having the highest statistics on infection and mortality rates. Regardless of existing sophisticated technologies and medical science, the spread has continued to surge high. With this COVID-19 Management System, organizations can respond virtually to the COVID-19 pandemic and protect, educate and care for citizens in the community in a quick and effective manner. This comprehensive solution not only helps in containing the virus but also proactively empowers both citizens and care providers to minimize the spread of the virus through targeted strategies and education.
Sachpazis_Consolidation Settlement Calculation Program-The Python Code and th...Dr.Costas Sachpazis
Consolidation Settlement Calculation Program-The Python Code
By Professor Dr. Costas Sachpazis, Civil Engineer & Geologist
This program calculates the consolidation settlement for a foundation based on soil layer properties and foundation data. It allows users to input multiple soil layers and foundation characteristics to determine the total settlement.
Hot Call Girls In Bangalore ✔ 9079923931 ✔ Hi I Am Divya Vip Call Girl Servic...
Ds practical file
1. DISTRIBUTED SYSTEMS | 1
PRACTICAL 1
OBJECTIVE: Simulate the functioning of Lamport’s Logical Clock in C.
LAMPORT LOGICAL CLOCK-
Leslie Lamport proposed this scheme to provide ordering of events in a distributed
environment using logical clocks. Because it is impossible to have perfectly synchronized
clocks and global time in a distributed system, it is often necessary to use logical clocks
instead.
Definitions:
1) Happened Before Relation (->). This relation captures causal dependencies between
events, that is, whether or not events have a cause and effect relation. This relation
(->) is defined as follows:
a -> b, if a and b are in the same process and a occurred before b.
a -> b, if a is the event of sending a message and b is the receipt of that
message by another process.
If a -> b and b -> c, then a -> c - that is, the relation has the property
of transitivity.
2) Causally Related Events: If event a -> event b, then a causally affects b.
3) Concurrent Events: Two distinct events a and b are concurrent (a || b) if (not) a -> b
And (not) b -> a. That is, the events have no causal relationship. This is equivalent to
b || a.
For any two events a and b in a system, only one of the following is true: a -> b, b -> a, or
a || b.
Lamport introduced a system of logical clocks in order to make the -> relation possible.
It works like this: Each process Pi in the system has its own clock Ci. Ci can be looked at as
a function that assigns a number, Ci(a) to an event a. This is the timestamp of the event a
in process Pi. These numbers are not in any way related to physical time -- that is why
they are called logical clocks. These are generally implemented using counters, which
increase each time an event occurs. Generally, an event's timestamp is the value of the
clock at that time it occurs.
Conditions Satisfied by the Logical Clock system
For any events a and b, if a -> b, then C(a) < C(b). This is true if two conditions are met:
If a occurs before b, then Ci(a) < Ci(b).
If a is a message sent from Pi and b is the receive of that same message in Pj,
then Ci(a) < Cj(b).
Implementation Rules Required-
Clock Ci is incremented for each event: Ci := Ci + d (d > 0)
If a is the event of sending a message from one process to another, then the
receiver sets its clock to the max of its current clock and the sender's clock –
that is, Cj := max(Cj, tm + d) (d > 0) .
2. DISTRIBUTED SYSTEMS | 2
C CODE
#include<stdio.h>
#include<conio.h>
long p1(int);
long p2(int);
long p3(int);
long p4(int);
void main() { int k;
clrscr();
printf("Enter process no.");
scanf("%d",&k);
while(!kbhit()) { if(k==1) p1(1);
if(k==2) p2(1);
if(k==3) p3(1);
if(k==4) p4(1);
}
getch();
printf("n Logical Clockn");
printf("P1:%ldnP2:%ldnP3:%ldnP4:%ldn",p1(0),p2(0),p3(0),p4(0));
getch();
}
long p1(int i) { static long a=0;
if(i==1) { a++; p2(1); return 1; }
else return a;
}
long p2(int i) { static long b=0;
if(i==1) { b++; p3(1); p4(1); return 1; }
else return b;
}
long p3(int i) { static long c=0;
if(i==1) { c++; return 1; }
else return c;
}
long p4(int i) { static long d=0;
if(i==1) { d++; p3(1); return 1; }
else return d;
}
3. DISTRIBUTED SYSTEMS | 3
PRACTICAL 2
OBJECTIVE: Simulate the functioning of Vector Clock in C .
VECTOR CLOCK-
Vector clock is an algorithm for generating a partial ordering of events in a distributed
system and detecting causality violations. Just as in Lamport timestamps, interprocess
messages contain the state of the sending process's logical clock.
Example of a system of vector clocks
A vector clock of a system of N processes is an array of N logical clocks, one per process,
a local copy of which is kept in each process with the following rules for clock updates:
Initially all clocks are zero.
Each time a process experiences an internal event, it increments its own logical
clock in the vector by one.
Each time a process prepares to send a message, it increments its own logical clock in
the vector by one and then sends its entire vector along with the message being sent.
Each time a process receives a message, it increments its own logical clock in the
vector by one and updates each element in its vector by taking the maximum of the
value in its own vector clock and the value in the vector in the received message (for
every element).
The vector clocks algorithm was independently developed by Colin Fidge and
Friedemann Mattern in 1988.
C CODE
#include<stdio.h>
#include<conio.h>
#include<stdio.h>
#include<stdlib.h>
long *p1(int i,long *comp);
long *p2(int i,long *comp);
4. DISTRIBUTED SYSTEMS | 4
long *p3(int i,long *comp);
void main()
{ long start[]={0,0,0},*vector;
clrscr();
while(!kbhit()) p1(1,&start[0]);
printf("n Process Vectorn");
vector=p1(0,&start[0]);
printf("p1[%ld%ld%ld]n",*vector,*(vector+1),*(vector+2));
vector=p2(0,&start[0]);
printf("p2[%ld%ld%ld]n",*vector,*(vector+1),*(vector+2));
vector=p3(0,&start[0]);
printf("p3[%ld%ld%ld]n",*vector,*(vector+1),*(vector+2));
getch();
}
long *p1(int i,long *comp) { static long a[]={0,0,0};
int next;
if(i==1){ if(*(comp+1)>a[1]) a[1]=*(comp+1);
if(*(comp+2)>a[2]) a[2]=*(comp+2);
next=random(2);
if(next==0) p2(1,&a[0]);
else if(next==1) p3(1,&a[0]);
return(&a[0]);
}
else return(&a[0]);
}
long *p2(int i,long *comp) { static long b[]={0,0,0};
int next;
if(i==1){ b[i]++;
if(*comp>b[0]) b[0]=*(comp);
if(*(comp+2)>b[2]) b[2]=*(comp+2);
next=random(2);
if(next==0) p1(1,&b[0]);
else if(next==1) p3(1,&b[0]);
return &b[0];
}
else return &b[0];
}
long *p3(int i,long *comp) { static long c[]={0,0,0};
int next;
if(i==1){ c[2]++;
if(*comp>c[0]) c[0]=*(comp);
if(*(comp+1)>c[1]) c[1]=*(comp+1);
next=random(2);
if(next==0) p1(1,&c[0]);
return &c[0];
}
else return &c[0];
}
5. DISTRIBUTED SYSTEMS | 5
PRACTICAL 3
OBJECTIVE: Simulate the Distributed Mutual Exclusion.
DISTRIBUTED MUTUAL EXCLUSION-
Systems involving multiple processes are often most easily programmed using critical
regions. When a process has to read or update certain shared data structures, it first
enters a critical region to achieve mutual exclusion and ensures that no other process will
use the shared data structures at the same time.
The mutual exclusion problem states that only a single process can be allowed access to a
protected resource, also termed as a Critical Section (CS), at any time. Mutual exclusion
is a form of synchronization and is one of the most fundamental paradigms in computing
systems.
Distributed mutual exclusion algorithms are divided in three basic categories:
1) Token based Algorithms
In the token-based approach, a unique token (also known as the PRIVILEGE message) is
shared among the sites. A site is allowed to enter its CS if it possesses the token and it
continues to hold the token until the execution of the CS is over. Mutual exclusion is
ensured because the token is unique. The algorithms based on this approach essentially
differ in the way a site carries out the search for the token.
2) Non token based Algorithms
In the non-token-based approach, two or more successive rounds of messages are
exchanged among the sites to determine which site will enter the CS next. A site enters
the critical section (CS) when an assertion, defined on its local variables, becomes true.
Mutual exclusion is enforced because the assertion becomes true only at one site at any
given time.
3) Quorum based Algorithms
In the quorum-based approach, each site requests permission to execute the CS from a
subset of sites (called a quorum). The quorums are formed in such a way that when two
sites concurrently request access to the CS, at least one site receives both the requests
and this site is responsible to make sure that only one request executes the CS at any
time.
JAVA CODE
import Utilities.*;
import Synchronization.*;
class Message { public int number, id;
public Message(int number, int id)
{ this.number = number; this.id = id;
}
}
6. DISTRIBUTED SYSTEMS | 6
class Node extends MyObject implements Runnable
{ private static final int MAIN = 0, REQUESTS = 1, REPLIES = 2;
private int whichOne = 0;
private int id = -1;
private int numNodes = -1;
// Both are in milliseconds
private int napOutsideCS = 0;
private int napInsideCS = 0;
private MessagePassing[] requestChannel = null;
private MessagePassing[] replyChannel = null;
private MessagePassing requestsToMe = null;
private MessagePassing repliesToMe = null;
private int number = 0;
private int highNumber = 0;
private boolean requesting = false;
private int replyCount = 0;
private BinarySemaphore s = new BinarySemaphore(1);
private BinarySemaphore wakeUp = new BinarySemaphore(0);
private boolean[] deferred = null;
public Node (String name, int id, int numNodes, int napOutsideCS, int napInsideCS,
MessagePassing[] requestChannel, MessagePassing replyChannel[],
MessagePassing requestsToMe, MessagePassing repliesToMe)
{ super(name + " " + id);
this.id = id;
this.numNodes = numNodes;
this.napOutsideCS = napOutsideCS;
this.napInsideCS = napInsideCS;
this.requestChannel = requestChannel;
this.replyChannel = replyChannel;
this.requestsToMe = requestsToMe;
this.repliesToMe = repliesToMe;
deferred = new boolean[numNodes];
for (int i = 0; i < numNodes; i++) deferred[i] = false;
System.out.println(getName() + " is alive, napOutsideCS=" + napOutsideCS +
", napInsideCS=" + napInsideCS);
new Thread(this).start();
}
public void run()
{ // Start three different threads in the same object
int meDo = whichOne++;
if (meDo == MAIN) { new Thread(this).start();
main();
}
else if (meDo == REQUESTS) { new Thread(this).start();
handleRequests();
}
else if (meDo == REPLIES) handleReplies();
}
8. DISTRIBUTED SYSTEMS | 8
sendRequest(); // PRE-PROTOCOL
waitForReply(); // PRE-PROTOCOL
insideCS();
System.out.println("age()=" + age() + ", node " + id
+ " has now left its critical
section");
replyToDeferredNodes(); // POST-PROTOCOL
}
}
private void handleRequests()
{ while (true)
{ Message m = (Message) receive(requestsToMe);
int receivedNumber = m.number;
int receivedID = m.id;
highNumber=Math.max(highNumber,receivedNumber);
P(s);
boolean decideToDefer = requesting && (number <
receivedNumber || (number == receivedNumber && id <
receivedID));
if (decideToDefer) deferred[receivedID] = true;
else send(replyChannel[receivedID], id);
V(s);
}
}
private void handleReplies()
{ while (true)
{ int receivedID = receiveInt(repliesToMe);
replyCount++;
if (replyCount == numNodes - 1) V(wakeUp);
}
}
}
class DistributedMutualExclusion extends MyObject
{ public static void main(String[] args)
{ // Parse command line options, if any, to override defaults
GetOpt go = new GetOpt(args, "Un:R:");
String usage = "Usage: -n numNodes -R runTime" + " napOutsideCS[i]
napInsideCS[i] i=0,1,...";
go.optErr = true;
int ch = -1;
int numNodes = 5;
int runTime = 60; // seconds
while ((ch = go.getopt()) != go.optEOF)
{ if((char)ch == 'U') { System.out.println(usage);
System.exit(0);}
9. DISTRIBUTED SYSTEMS | 9
else if ((char)ch == 'n') numNodes=go.processArg (go.optArgGet(),
numNodes);
else if ((char)ch == 'R') runTime = go.processArg (go.optArgGet(),
runTime);
else { System.err.println(usage);
System.exit(1);
}
}
System.out.println("DistributedMutualExclusion: numNodes=" + numNodes +
", runTime=" + runTime);
// process non-option command line arguments
int[] napOutsideCS = new int[numNodes];
int[] napInsideCS = new int[numNodes];
int argNum = go.optIndexGet();
for (int i = 0; i < numNodes; i++)
{ napOutsideCS[i]=go.tryArg(argNum++, 8);
napInsideCS[i] = go.tryArg(argNum++, 2);
}
// create communication channels
MessagePassing[] requestChannel = null, replyChannel = null,
requestChannelS = null, requestChannelR = null,
replyChannelS = null, replyChannelR = null;
requestChannel = new MessagePassing[numNodes];
replyChannel = new MessagePassing[numNodes];
requestChannelS = new MessagePassing[numNodes];
replyChannelS = new MessagePassing[numNodes];
requestChannelR = new MessagePassing[numNodes];
replyChannelR = new MessagePassing[numNodes];
for (int i = 0; i < numNodes; i++)
{ requestChannel[i]=new AsyncMessagePassing();
replyChannel[i] = new AsyncMessagePassing();
requestChannelS[i] = new MessagePassingSendOnly
(requestChannel[i]);
replyChannelS[i] = new MessagePassingSendOnly(replyChannel[i]);
requestChannelR[i] = new MessagePassingReceiveOnly
(requestChannel[i]);
replyChannelR[i] = new MessagePassingReceiveOnly(replyChannel[i]);
}
// create the Nodes (they start their own threads)
for (int i = 0; i < numNodes; i++)
{ new Node("Node", i, numNodes, napOutsideCS[i]*1000,
napInsideCS[i]*1000, requestChannelS, replyChannelS,
requestChannelR[i], replyChannelR[i]);
}
System.out.println("All Nodes created");
// let the Nodes run for a while
10. DISTRIBUTED SYSTEMS | 10
nap(runTime*1000);
System.out.println("age()=" + age() + ", time to stop the threads & exit");
System.exit(0);
}
}
OUTPUT
D:JavaAdvanced>javac dimu.java
D:JavaAdvanced >java DistributedMutualExclusion -R20
DistributedMutualExclusion: numNodes=5, runTime=20
Node 0 is alive, napOutsideCS=8000, napInsideCS=2000
Node 1 is alive, napOutsideCS=8000, napInsideCS=2000
Node 2 is alive, napOutsideCS=8000, napInsideCS=2000
Node 3 is alive, napOutsideCS=8000, napInsideCS=2000
Node 4 is alive, napOutsideCS=8000, napInsideCS=2000
age()=170, Node 1 napping outside CS for 2719 ms
age()=170, Node 2 napping outside CS for 279 ms
All Nodes created
age()=170, Node 3 napping outside CS for 2355 ms
age()=220, Node 0 napping outside CS for 2393 ms
age()=220, Node 4 napping outside CS for 8 ms
age()=220, node 4 wants to enter its critical section
age()=330, Node 4 napping inside CS for 911 ms
age()=440, node 2 wants to enter its critical section
age()=1260, node 4 has now left its critical section
age()=1260, Node 4 napping outside CS for 4042 ms
age()=1260, Node 2 napping inside CS for 183 ms
age()=1480, node 2 has now left its critical section
age()=1480, Node 2 napping outside CS for 7335 ms
age()=2530, node 3 wants to enter its critical section
age()=2530, Node 3 napping inside CS for 741 ms
age()=2580, node 0 wants to enter its critical section
age()=2860, node 1 wants to enter its critical section
age()=3300, node 3 has now left its critical section
age()=3300, Node 3 napping outside CS for 6849 ms
age()=3300, Node 0 napping inside CS for 1710 ms
age()=5000, node 0 has now left its critical section
age()=5000, Node 0 napping outside CS for 5253 ms
age()=5000, Node 1 napping inside CS for 1694 ms
age()=5330, node 4 wants to enter its critical section
age()=6700, node 1 has now left its critical section
age()=6700, Node 1 napping outside CS for 3063 ms
age()=6700, Node 4 napping inside CS for 397 ms
age()=7140, node 4 has now left its critical section
age()=7140, Node 4 napping outside CS for 3687 ms
age()=8790, node 2 wants to enter its critical section
age()=8790, Node 2 napping inside CS for 102 ms
11. DISTRIBUTED SYSTEMS | 11
age()=8900, node 2 has now left its critical section
age()=8900, Node 2 napping outside CS for 1174 ms
age()=9780, node 1 wants to enter its critical section
age()=9780, Node 1 napping inside CS for 1617 ms
age()=10110, node 2 wants to enter its critical section
age()=10160, node 3 wants to enter its critical section
age()=10270, node 0 wants to enter its critical section
age()=10820, node 4 wants to enter its critical section
age()=11430, node 1 has now left its critical section
age()=11430, Node 1 napping outside CS for 5326 ms
age()=11430, Node 2 napping inside CS for 628 ms
age()=12090, node 2 has now left its critical section
age()=12090, Node 2 napping outside CS for 4970 ms
age()=12090, Node 3 napping inside CS for 545 ms
age()=12630, node 3 has now left its critical section
age()=12630, Node 3 napping outside CS for 7989 ms
age()=12630, Node 0 napping inside CS for 904 ms
age()=13510, node 0 has now left its critical section
age()=13510, Node 0 napping outside CS for 4162 ms
age()=13510, Node 4 napping inside CS for 1440 ms
age()=15000, node 4 has now left its critical section
age()=15000, Node 4 napping outside CS for 2578 ms
age()=16750, node 1 wants to enter its critical section
age()=16750, Node 1 napping inside CS for 123 ms
age()=16860, node 1 has now left its critical section
age()=16860, Node 1 napping outside CS for 3709 ms
age()=17030, node 2 wants to enter its critical section
age()=17030, Node 2 napping inside CS for 97 ms
age()=17140, node 2 has now left its critical section
age()=17140, Node 2 napping outside CS for 7901 ms
age()=17580, node 4 wants to enter its critical section
age()=17580, Node 4 napping inside CS for 1695 ms
age()=17690, node 0 wants to enter its critical section
age()=19280, node 4 has now left its critical section
age()=19280, Node 4 napping outside CS for 3751 ms
age()=19280, Node 0 napping inside CS for 869 ms
age()=20160, node 0 has now left its critical section
age()=20160, Node 0 napping outside CS for 6489 ms
age()=20160, time to stop the threads and exit
... end of example run(s) */
12. DISTRIBUTED SYSTEMS | 12
PRACTICAL 4
OBJECTIVE: Implement a Distributed Chat Server using TCP Sockets.
DISTRIBUTED CHAT SERVER USING TCP SOCKETS-
A chat server allows clients to broadcast messages to other clients that have joined in the
chat session (called peers). The twist is that the chat server is effectively distributed
among all the clients that are currently running, with a central server used mainly to
bootstrap a client that wants to join the chat session, so that the client can find one (or
several) other clients.
DISTRIBUTED CHAT SERVICE
JAVA CODE
1)Server.java
import java.net.*; import java.io.*;
public class server { public static void main(String args[]) throws IOException
{ ServerSocket s1=null;
try { s1=new ServerSocket(98); }
catch(Exception e) { System.out.println("Port not found");
e.printStackTrace(); }
Socket c=null;
try { c=s1.accept();
System.out.println("Connection from"+c); }
catch(Exception e) { System.out.println("not accepted");
e.printStackTrace(); }
PrintWriter out=new PrintWriter(c.getOutputStream(),true);
BufferedReader in = new BufferedReader (new
InputStreamReader (c.getInputStream()));
13. DISTRIBUTED SYSTEMS | 13
String l;
BufferedReader sin = new BufferedReader (new
InputStreamReader (System.in));
System.out.println("I am ready type now");
while((l=sin.readLine())!=null) out.println(l);
out.close();
sin.close();
c.close(); s1.close();
}
}
-------------------------------------------------------------------------------------------------------------------------
2)Client.java
import java.net.*; import java.io.*;
public class Client { public static void main(String args[]) throws IOException
{ Socket s=null;
BufferedReader b=null;
try{ s=new Socket(InetAddress.getLocalHost(),98);
b=new BufferedReader(new InputStreamReader
(s.getInputStream())); }
catch(Exception e){ System.out.println("I do not host");
e.printStackTrace(); }
String inp;
while ((inp=b.readLine())!=null) System.out.println(inp);
b.close(); s.close();
}
}
OUTPUT
Running the application
Open two command prompt windows and follow these:
1) Java Server
D:JavaNetWorkingChatServer>java Server
Connection fromSocket[addr=/127.0.0.1,port=1120,localport=98]
I am ready type now
Hello how r u? dude…
2) Java client
D:JavaNetWorkingChatServer>java Client
Hello how r u? dude…
14. DISTRIBUTED SYSTEMS | 14
PRACTICAL 5
OBJECTIVE: Implement Remote Procedure Call (RPC) mechanism for a file
transfer across a network in C.
REMOTE PROCEDURE CALL (RPC)-
Remote procedure call (RPC) is an Inter-process communication technology that allows a
computer program to cause a subroutine or procedure to execute in another address
space (commonly on another computer on a shared network) without the programmer
explicitly coding the details for this remote interaction. That is, the programmer would
write essentially the same code whether the subroutine is local to the executing
program, or remote.
C CODE
class CDatabaseProtocol : public CProtocol
{ private: CClient Communication *m_Bothways;
public: CDatabaseProtocol(CClientCommunication *);
BOOL Open (const char* pszFileName, UINT nOpenFlags, CFileException*
pError=NULL);
UINT Read (void FAR* lpBuf,UINT nCount);
void Write(const void FAR* lpBuf, UINT nCount); void Close(); };
…………………………………………………………………………………………………………..
class CClientDatabaseProtocol : public CDatabaseProtocol
{ public: CClientDatabaseProtocol(CClientCommunication *);
BOOL AddData(int *iIndex, CHAINLIST *clElement);
BOOL RemoveData(int *iIndex);
BOOL RetrieveData(int *iIndex, CHAINLIST *cpResult);
BOOL GetEntries(int *);
BOOL Terminate(); };
…………………………………………………………………………………………………………..
#ifdef SERVER
class CServerDatabaseProtocol : public CDatabaseProtocol
{ public: CServerDatabaseProtocol(CServerCommunication *);
BOOL AcceptCommand(int *iCommand, CHAINLIST *cpElement, int *iIndex);
BOOL ProcessCommand(CHAINLIST *cpElement, int *iIndex);
BOOL Fail(int iErrorCode);
BOOL Acknowledge(int iIndex); };
…………………………………………………………………………………………………………..
// The AddData function works on the client side
BOOL CClientDatabaseProtocol : : AddData(int *iIndex, CHAINLIST *clElement)
{ int iData;
// First write the stuff out.
iData=CMD_ADDRECORD;
15. DISTRIBUTED SYSTEMS | 15
_try { Write((char *)&iData,sizeof(int))
Write((char *)&clElement->iSecuredElement,sizeof(int));
Write((char *)&clElement->iInsecuredElement,sizeof(int));
// Then wait for the response
Read((char *)&iData,sizeof(int));
if(iData==CMD_SUCCESS){ Read((char *)iIndex,sizeof(int));
return TRUE; }
else { Read((char *)&m_iErrorCode,sizeof(int));
return FALSE; } };
_except (EXCEPTION_EXECUTE_HANDLER)
{ m_iErrorCode=GetExceptionCode; };
return FALSE; };
…………………………………………………………………………………………………………..
// The server side of transaction
BOOL CServerDatabaseProtocol : : AcceptCommand(int *iCommand, CHAINLIST
*cpElement, int *iIndex);
{ _try { if(Read((char *)iCommand,sizeof(int))!=sizeof(int)) return FALSE;
//Fetch the command first
switch(*iCommand)
{ case CMD_ADDRECORD:
if(Read((char *)&cpElement->iSecuredElement,sizeof(int))!=sizeof(int)) return FALSE;
if(Read((char *)&cpElement->iInsecuredElement,sizeof(int))!=sizeof(int)) return FALSE;
return TRUE;
case CMD_DELETERECORD:
if(Read((char *)iIndex,sizeof(int))!=sizeof(int)) return FALSE;
return TRUE;
case CMD_RETRIEVERECORD:
if(Read((char *)iIndex,sizeof(int))!=sizeof(int)) return FALSE;
return TRUE;
case CMD_GETENTRIES: return TRUE;
}; // Switch
return TRUE; };
_except(EXCEPTION_EXECUTE_HANDLER)
{ m_iErrorCode=GetExceptionCode; };
return FALSE; };
…………………………………………………………………………………………………………..
// AcceptCommand use
long WINAPI PipeThreadFunction(CNpscervView *cvTarget)
{ char szDiagnosticMessage[255];
CServerDatabaseProtocol *cpProt;
BOOL bFinished;
ServerChainedQueue *cqTheQueue = cvTarget->m_cqQueue;
if(!cvTarget->m_cpPipe->AwaitCommunicationAttempt()) {
cvTarget->DisplayTextErrorMessage(“Open named pipe failed --%s”,
cvTarget->m_cpPipe->m_iErrorCode);
goto ErrorExit; }
else { sprintf(szDiagnosticMessage,”Open named pipe succeeded”);
cvTarget->AddStringandAdjust(szDiagnosticMessage); }
// Acknowledge communication to the UI
cpProt=new CServerDatabaseProtocol(cvTarget->m_cpPipe);
if(!cpProt->Open(“”,CFile::modeReadWrite)) // We are server….
{ //Log an error here
goto ErrorExit;
17. DISTRIBUTED SYSTEMS | 17
PRACTICAL 6
OBJECTIVE: Implement Java Remote Method Invocation (RMI) mechanism
for accessing methods of remote systems.
JAVA REMOTE METHOD INVOCATION (RMI)-
The Java Remote Method Invocation Application Programming Interface, or Java RMI, is
a Java application programming interface that performs the object-oriented equivalent
of remote procedure calls (RPC).
1. The original implementation depends on Java Virtual Machine (JVM) class
representation mechanisms and it thus only supports making calls from one JVM to
another. The protocol underlying this Java-only implementation is known as Java
Remote Method Protocol (JRMP).
2. In order to support code running in a non-JVM context, a CORBA version was later
developed.
Usage of the term RMI may denote solely the programming interface or may signify both
the API and JRMP, whereas the term RMI-IIOP (read: RMI over IIOP) denotes the RMI
interface delegating most of the functionality to the supporting CORBA implementation.
The programmers of the original RMI API generalized the code somewhat to support
different implementations, such as a HTTP transport. Additionally, the ability to pass
arguments "by value" was added to CORBA in order to support the RMI interface. Still, the
RMI-IIOP and JRMP implementations do not have fully identical interfaces.
RMI functionality comes in the package java.rmi, while most of Sun's implementation is
located in the sun.rmi package. Note that with Java versions before Java 5.0 developers
had to compile RMI stubs in a separate compilation step using rmic. Version 5.0 of Java
and beyond no longer require this step.
Jini offers a more advanced version of RMI in Java. It functions similarly but provides
more advanced searching capabilities and mechanisms for distributed object applications.
A typical implementation model of Java-RMI using stub and skeleton objects.
Java 2 SDK, Standard Edition, v1.2 removed the need for a skeleton.
18. DISTRIBUTED SYSTEMS | 18
JAVA CODE
1) INTERFACES: The first step is to write & compile the Java code for the service
interface. The Calculator interface defines all of the remote features offered by the
service:
Calculator.java
public interface Calculator extends java.rmi.Remote {
public long add(long a, long b) throws java.rmi.RemoteException;
public long sub(long a, long b) throws java.rmi.RemoteException;
public long mul(long a, long b) throws java.rmi.RemoteException;
public long div(long a, long b) throws java.rmi.RemoteException; }
…………………………………………………………………………………………………………..
2) IMPLEMENTATION: Next, you write the implementation for the remote service.
This is the CalcualtorImpl class:
CalculatorImpl.java
public class CalculatorImpl
extends java.rmi.server.UnicastRemoteObject implements Calculator {
// Implementations must have an explicit constructor in order to declare the
RemoteException exception
public CalculatorImpl() throws java.rmi.RemoteException { super(); }
public long add(long a, long b) throws java.rmi.RemoteException
{ return a + b; }
public long sub(long a, long b) throws java.rmi.RemoteException
{ return a - b; }
public long mul(long a, long b) throws java.rmi.RemoteException
{ return a * b; }
public long div(long a, long b) throws java.rmi.RemoteException
{ return a / b; } }
…………………………………………………………………………………………………………..
3) STUBS & SKELETONS: You next use the RMI Compiler, rmic, to generate the
stub & skeleton files. The compiler runs on the remote service implementation class
file. The Java 2 platform version of rmic add three new options:
-v1.1 Create stubs/skeletons for JDK 1.1 stub protocol version only
-vcompat (default) Create stubs/skeletons ompatible with JDK 1.1 & Java 2
-v1.2 Create stubs for Java 2 stub protocol version only
…………………………………………………………………………………………………………..
4) HOST SERVER: Remote RMI Services must be hosted in a server process. The class
CalculatorServer is a very simple server that provides the bare essentials for hosting.
CalculatorServer.java
import java.rmi.Naming;
public class CalculatorServer {
public CalculatorServer() {
try { Calculator c = new CalculatorImpl();
Naming.rebind("rmi://localhost:1099/CalculatorService", c);}
19. DISTRIBUTED SYSTEMS | 19
catch (Exception e) { System.out.println("Trouble: " + e); } }
public static void main(String args[]) { new CalculatorServer(); } }
…………………………………………………………………………………………………………..
5) CLIENT: The source code for the client follows:
CalculatorClient.java
import java.rmi.Naming;
import java.rmi.RemoteException;
import java.net.MalformedURLException;
import java.rmi.NotBoundException;
public class CalculatorClient {
public static void main(String[] args) {
try { Calculator c = (Calculator)
Naming.lookup("rmi://localhost/CalculatorService");
System.out.println( c.sub(4, 3) );
System.out.println( c.add(4, 5) );
System.out.println( c.mul(3, 6) );
System.out.println( c.div(9, 3) );
}
catch (MalformedURLException murle) { System.out.println();
System.out.println("MalformedURLExceptio
n");
System.out.println(murle); }
catch (RemoteException re) { System.out.println();
System.out.println("RemoteException");
System.out.println(re); }
catch (NotBoundException nbe) { System.out.println();
System.out.println("NotBoundException");
System.out.println(nbe); }
catch ( java.lang.ArithmeticException ae) { System.out.println();
System.out.println("java.lang.ArithmeticEx
ception");
System.out.println(ae); }
}
}
OUTPUT
Running the Application-
D:JavaNetWorkingRMI>rmic CalculatorImpl
Now open three command prompt windows and follow these at each:
1. D:JavaNetWorkingRMI>Rmiregistry
2. D:JavaNetWorkingRMI>java CalculatorServer
3. D:JavaNetWorkingRMI>java CalculatorClient
If all goes well you will see the following output:
1 9 18 3
20. DISTRIBUTED SYSTEMS | 20
PRACTICAL 7
OBJECTIVE: Simulate Balanced Sliding Window Protocol in C
BALANCED SLIDING WINDOW PROTOCOL-
This is a symmetric protocol that allows information to be sent reliably in both directions.
The FIFO property of channels is assumed as it is used for exchange of information
between stations that are directly connected through a line.
C CODE
#include <cisePort.h>
#include <sim.h>
#include <component.h>
#include <comptypes.h>
#include <list.h>
#include <eventdefs.h>
#include <main.h>
#include <route_activity.h>
#include <sim_tk.h>
#include <dlc_layer.h>
21. DISTRIBUTED SYSTEMS | 21
/* Sequence number manipulation/ test macros : Accounts for Data Availability in buffer and
wrap around */
#define IncerementSequenceNumber(i,N)
{ (i)=((i)+1)%(N); }
#define out_of_sequence_pdu(pdu,dci)
(pdu->u.d_pdu.seq_number != dci->rcv_nxt)
…………………………………………………………………………………………………………..
static int window_open(DLC_Conn_Info_TYPE *dci)
{ int result;
int data_available=DataInPDUBuffer(dci);
int occupied_window;
occupied_window=((dci->snd_nxt>=dci->snd_una)? (dci->snd_nxt – dci->snd_una) :
(dci->snd_nxt + dci->window_size + 1 – dci->snd_una));
result=((occupied_window < data_available) && (occupied_window < dci-
>window_size));
return result; }
…………………………………………………………………………………………………………..
static dlc_layer_receive (DLC_LAYER_ENTITY_TYPE *dlc_layer_entity,
GENERIC_LAYER_ENTITY *generic_layer_entity, PDU_TYPE *pdu)
{ DLC_Conn_Info_TYPE *dci;
dci=Datalink_Get_Conn_Info(dlc_layer_entity,pdu);
/* Gets the appropriate DLC_Conn_Info_TYPE structure */
if(DatalinkFromApplication(generic_layer_entity))
{ // Insert A_PDU into dci_buf
InsertPDUIntoBuffer(dlc_layer_entity,pdu,dci);
// Sends from the buffer
AttemptToSend(dlc_layer_entity,dci); }
else if(DatalinkFromPhysical(generic_layer_entity))
{ if(pdu->u.d_pdu.error==YES)
DatalinkProcessError(dlc_layer_entity,pdu,dci);
else if(pdu->u.d_pdu.type==D_ACK)
DatalinkProcessACK(dlc_layer_entity,pdu,dci);
else if(pdu->u.d_pdu.type==D_NAK)
DatalinkProcessNAK(dlc_layer_entity,pdu,dci);
else if(pdu->u.d_pdu.type==D_INFO)
DatalinkProcessInfo(dlc_layer_entity,pdu,dci); }
return 0; }
…………………………………………………………………………………………………………..
/* DO YOUR CODING HERE */
…………………………………………………………………………………………………………..
static DatalinkProcessError(DLC_LAYER_ENTITY_TYPE *dlc_layer_entity, PDU_TYPE
*pdu, DLC_Conn_Info_TYPE *dci)
{ /* Simply free PDU */
/* Optional : Send NAK */
return 0; }
…………………………………………………………………………………………………………..
22. DISTRIBUTED SYSTEMS | 22
static DatalinkProcessACK(DLC_LAYER_ENTITY_TYPE *dlc_layer_entity, PDU_TYPE
*pdu, DLC_Conn_Info_TYPE *dci)
{ /* Free up space in the retransmission buffer */
/* Use UpdatePDUBuffer(); */
/* Update snd_una */
/* Send as many pdu’s as allowed by window */
/* Use window_open() and AttemptToSend() */
/* Free pdu */
return 0; }
…………………………………………………………………………………………………………..
static DatalinkProcessNAK(DLC_LAYER_ENTITY_TYPE *dlc_layer_entity, PDU_TYPE
*pdu, DLC_Conn_Info_TYPE *dci)
{ /* Free up space in the retransmission buffer because a NAK may ack a few PDUs*/
/* Set snd_una and snd_next */
/* Send as many pdu’s as allowed by window */
/* Use window_open() and AttemptToSend() */
/* Free pdu */
return 0; }
…………………………………………………………………………………………………………..
static DatalinkProcessInfo(DLC_LAYER_ENTITY_TYPE *dlc_layer_entity, PDU_TYPE
*pdu, DLC_Conn_Info_TYPE *dci)
{ PDU_TYPE *pdu_to_application;
/* OutOfSequence PDU => send NAK, discard pdu and return 0 */
/* Use out_of_sequence_pdu() and SendNak() */
/* Expected PDU => Increment rcv_nxt */
/* Use a maximum sequence of one more than window size */
/* Use IncrementSequenceNumber(dci->rcv_nxt,(dci->window_size + 1)); */
/* Reset nak_already_sent. Ack received => Naks maybe sent */
/* Send an Ack. Use SendAck(); */
/* You can ignore piggybacked Acks if you want. If you choose to ignore them, then
make sure you send non piggybacked Acks */
/* --- Send pdu to application : Same as Lab1 --- */
/* --- Send to app --- */
send_pdu_to_application_layer(dlc_layer_entity, pdu_to_application);
pdu_free(pdu);
return 0; }
…………………………………………………………………………………………………………..
/* Do not change the name of the following function */
/* This function is automatically called when the timer expires */
static DatalinkTimerExpired(DLC_LAYER_ENTITY_TYPE *dlc_layer_entity,
DLC_Conn_Info_TYPE *dci)
{ dci->snd_nxt = dci->snd_una;
/* Retransmit all unacknowledged D_PDUs */
/* Send as many pdu’s as allowed by window */
/* Use window_open() and AttemptToSend() */
return 0; }
…………………………………………………………………………………………………………..
23. DISTRIBUTED SYSTEMS | 23
/* --------- Similar to former DatalinkToPhysical --------- *
static AttemptToSend(DLC_LAYER_ENTITY_TYPE *dlc_layer_entity,
DLC_Conn_Info_TYPE *dci)
{ PDU_TYPE *pdu_to_send;
PDU_TYPE *pdu_to_physical = pdu_alloc();
if(window_open(dci)) { /* Set a retransmission timer */
/* Use SetTimer(); */
/* Get PDU from buffer, copy it to pdu_to_physical
and fill the remaining fields of pdu_to_physical */
/* send_pdu_to_physical_layer(); */
/* Increment snd_nxt */ }
return 0; }
…………………………………………………………………………………………………………..
static SendAck(DLC_LAYER_ENTITY_TYPE *dlc_layer_entity, PDU_TYPE *pdu,
DLC_Conn_Info_TYPE *dci)
{ PDU_TYPE *pdu_to_physical=pdu_alloc();
/* Fill in the neede fields */
/* Send to physical layer */
return 0; }
…………………………………………………………………………………………………………..
static SendNak(DLC_LAYER_ENTITY_TYPE *dlc_layer_entity, PDU_TYPE *pdu,
DLC_Conn_Info_TYPE *dci)
{ PDU_TYPE *pdu_to_physical;
/* Don’t send Nak if nak_already_sent is 1, but do send an ack */
pdu_to_physical=pdu_alloc();
/* Fill in the needed fields */
/* Send to physical layer */
/* Set nak_already_sent to 1 */
return 0; } }
24. DISTRIBUTED SYSTEMS | 24
PRACTICAL 8
OBJECTIVE: Implement Common Object Request Broker Architecture
(CORBA) mechanism by using Java program.
COMMON OBJECT REQUEST BROKER ARCHITECTURE
(CORBA)
The Common Object Request Broker Architecture (CORBA) is a standard defined by
the Object Management Group (OMG) that enables software components written in
multiple computer languages and running on multiple computers to work together, i.e. it
supports multiple platforms.
Illustration of the autogeneration of the infrastructure code from an interface defined using the CORBA IDL
CORBA aims to bring to the table many benefits that no other single technology brings in
one package. These benefits include language- and OS-independence, freedom from
technology-linked implementations, strong data-typing, high level of tunability, and
freedom from the details of distributed data transfers.
JAVA CODE
1) FileInterface.idl
interface FileInterface { typedef sequence<octet> Data;
Data downloadFile(in string fileName); };
Now, let's compile the FileInterface.idl and generate server-side skeletons. Using the
command:
D:JavaCORBA> idlj -fserver FileInterface.idl
…………………………………………………………………………………………………………..
25. DISTRIBUTED SYSTEMS | 25
2) FileServant.java
import java.io.*;
public class FileServant extends _FileInterfaceImplBase
{ public byte[] downloadFile(String fileName) {
File file = new File(fileName);
byte buffer[] = new byte[(int)file.length()];
try { BufferedInputStream input = new BufferedInputStream (new
FileInputStream (fileName));
input.read(buffer,0,buffer.length);
input.close(); }
catch(Exception e) { System.out.println("FileServant Error: " +
e.getMessage() );
e.printStackTrace(); }
return(buffer); } }
…………………………………………………………………………………………………………..
3) FileServer.java
import java.io.*;
import org.omg.CosNaming.*;
import org.omg.CosNaming.NamingContextPackage.*;
import org.omg.CORBA.*;
public class FileServer { public static void main(String args[]) {
try { // create and initialize the ORB
ORB orb = ORB.init(args, null);
// create the servant and register it with the ORB
FileServant fileRef = new FileServant();
orb.connect(fileRef);
// get the root naming context
org.omg.CORBA.Object objRef =
orb.resolve_initial_references("NameService");
NamingContext ncRef = NamingContextHelper.narrow(objRef);
// Bind the object reference in naming
NameComponent nc = new NameComponent("FileTransfer", " ");
NameComponent path[] = {nc};
ncRef.rebind(path, fileRef);
System.out.println("Server started....");
// Wait for invocations from clients
java.lang.Object sync = new java.lang.Object();
synchronized(sync) { sync.wait(); } }
catch(Exception e) { System.err.println("ERROR: " + e.getMessage());
e.printStackTrace(System.out); } }
}
…………………………………………………………………………………………………………..
4) FileClient.java
import java.io.*;
import java.util.*;
import org.omg.CosNaming.*;
import org.omg.CORBA.*;
public class FileClient { public static void main(String argv[]) {
try { // create and initialize the ORB
ORB orb = ORB.init(argv, null);
26. DISTRIBUTED SYSTEMS | 26
// get the root naming context
org.omg.CORBA.Object objRef =
orb.resolve_initial_references("NameService");
NamingContext ncRef = NamingContextHelper.narrow(objRef);
NameComponent nc = new NameComponent("FileTransfer", " ");
// Resolve the object reference in naming
NameComponent path[] = {nc};
FileInterfaceOperations fileRef =
FileInterfaceHelper.narrow(ncRef.resolve(path));
if(argv.length < 1)
System.out.println("Usage: java FileClient filename");
// save the file
File file = new File(argv[0]);
byte data[] = fileRef.downloadFile(argv[0]);
BufferedOutputStream output = new
BufferedOutputS
tream(new FileOutputStream(argv[0]));
output.write(data, 0, data.length);
output.flush();
output.close(); }
catch(Exception e) { System.out.println("FileClient Error: " + e.getMessage());
e.printStackTrace(); } } }
OUTPUT
Running the application
1. D:JavaCORBA>tnameserv
2. D:JavaCORBA>java FileServer
3. D:JavaCORBA>idlj -fclient FileInterface.idl
4. D:JavaCORBA>java FileClient hello.txt
27. DISTRIBUTED SYSTEMS | 27
INDEX
S.NO. PRACTICAL TITLE PAGE
NO.
DATE SIGNATURE
1.
Simulate the functioning of
Lamport’s Logical Clock in
C.
1-2
2.
Simulate the functioning of
Vector Clock in C .
3-4
3.
Simulate the Distributed
Mutual Exclusion.
5-11
4.
Implement a Distributed
Chat Server using TCP
Sockets.
12-13
5.
Implement Remote
Procedure Call (RPC)
mechanism for a file
transfer across a network
in C.
14-16
6.
Implement Java Remote
Method Invocation (RMI)
mechanism for accessing
methods of remote
systems.
17-19
7.
Simulate Balanced Sliding
Window Protocol in C
20-23
8.
Implement Common Object
Request Broker
architecture using Java
program.
24-26