PARALLEL AND CLUSTER COMPUTING

Concepts and Terminology; von Neumann Computer Architecture, Flynn's Classical Taxonomy, general Parallel Terminology. Parallel Computer Memory Architectures; Shared Memory, Distributed Memory, Hybrid Distributed-Shared Memory. Parallel Programming Models; Shared Memory Model, Threads Model, Message Passing Model, Data Parallel Model, Other Models. Designing Parallel Programs; Automatic vs. Manual Parallelization, Partitioning, Communications, Synchronization, Data Dependencies, Load Balancing, Granularity, I/O, Limits and Costs of Parallel Programming, Performance Analysis and Tuning. Case Studies; Array Processing, PI Calculation, Simple Heat Equation, 1-D Wave Equation. Introduction to Cluster Computing; Dependable Clustered Computing. High Speed Networks. Lightweight Messaging Systems. Load Balancing over Network. Memory of Clusters: Network RAM; Distributed Shared Memory. Parallel I/O for Clusters: Methodologies and Systems. Introduction to Grids. Metacomputing: Harnessing Informal Supercomputers. Grid Computing: Technologies; Security; Information Services; Resource Management; Data Management; Applications.