Big Data Systems (WT 2020/21) - tele-TASK
The amount of data that can be generated and stored in academic and industrial projects and applications is increasing rapidly. Big data analytics technologies have established themselves as a solution for big data challenges to the scalability problems of traditional database systems. The vast amounts of new data that is collected, however, usually is not as easily analyzed as curated, structured data in a data warehouse is. Typically, these data are noisy, of varying format and velocity, and need to be analyzed with techniques from statistics and machine learning rather than pure SQL-like aggregations and drill-downs. Moreover, the results of the analyses frequently are models that are used for decision making and prediction. The complete process of big data analysis is described as a pipeline, which includes data recording, cleaning, integration, modeling, and interpretation. In this lecture, we will discuss big data systems, i.e., infrastructures that are used to handle all steps in typical big data processing pipelines. We will learn about data center infrastructure and scale-out software systems. The software discussed will cover the full big data stack, i.e., distributed file systems, Map Reduce, key value stores, stream processing, graph processing, ML systems.Podcasts
BigBench / TPCx-BB - Big Data Benchmark
Published: Feb. 1, 2021, 11:15 a.m.Duration: 13 minutes 48 seconds
Listed in: Education
Sort Benchmarks
Published: Feb. 1, 2021, 10:55 a.m.Duration: 16 minutes 25 seconds
Listed in: Education
Some Statistics
Published: Feb. 1, 2021, 9:55 a.m.Duration: 47 minutes 45 seconds
Listed in: Education
Measurements & Metrics
Published: Feb. 1, 2021, 9:40 a.m.Duration: 14 minutes 22 seconds
Listed in: Education
Back of the Envelope Calculation
Published: Feb. 1, 2021, 9:25 a.m.Duration: 12 minutes 6 seconds
Listed in: Education
A Brief Introduction to RDMAs
Published: Jan. 26, 2021, 10:45 a.m.Duration: 22 minutes
Listed in: Education
Intro to Persistent Memory II
Published: Jan. 26, 2021, 10:25 a.m.Duration: 20 minutes 25 seconds
Listed in: Education
Intro to Persistent Memory I
Published: Jan. 26, 2021, 10:15 a.m.Duration: 10 minutes 23 seconds
Listed in: Education
Data Processing on GPUs
Published: Jan. 26, 2021, 9:35 a.m.Duration: 37 minutes 26 seconds
Listed in: Education
Modern Hardware I
Published: Jan. 26, 2021, 9:15 a.m.Duration: 19 minutes 31 seconds
Listed in: Education
Data-Parallel Parameter Server
Published: Jan. 18, 2021, 11:55 a.m.Duration: 22 minutes 5 seconds
Listed in: Education
Execution Strategies
Published: Jan. 18, 2021, 11:10 a.m.Duration: 44 minutes 8 seconds
Listed in: Education
Language Abstraction & System Architectures
Published: Jan. 18, 2021, 10:20 a.m.Duration: 18 minutes 46 seconds
Listed in: Education
ML System Stack
Published: Jan. 18, 2021, 9:50 a.m.Duration: 28 minutes 2 seconds
Listed in: Education
Machine Learning Models
Published: Jan. 18, 2021, 9:35 a.m.Duration: 15 minutes 58 seconds
Listed in: Education
Federated Machine Learning
Published: Jan. 18, 2021, 12:20 a.m.Duration: 13 minutes 48 seconds
Listed in: Education
Graph Databases
Published: Jan. 7, 2021, 9:15 a.m.Duration: 2 hours 59 minutes 18 seconds
Listed in: Education
Relational & Big-Data Processing in the Enterprise - Bridging the Gap
Published: Jan. 6, 2021, 9:15 a.m.Duration: 57 minutes 17 seconds
Listed in: Education
MapReduce II
Published: Nov. 30, 2020, 9:15 a.m.Duration: 13 minutes 12 seconds
Listed in: Education
(Big Data) File Format
Published: Nov. 25, 2020, 10:55 a.m.Duration: 13 minutes 57 seconds
Listed in: Education
Erasure Coding
Published: Nov. 25, 2020, 10:45 a.m.Duration: 9 minutes 46 seconds
Listed in: Education
Hadoop Distributed File System
Published: Nov. 25, 2020, 10:30 a.m.Duration: 11 minutes 3 seconds
Listed in: Education
Google File System
Published: Nov. 25, 2020, 10:10 a.m.Duration: 20 minutes 13 seconds
Listed in: Education
Network File System
Published: Nov. 25, 2020, 9:40 a.m.Duration: 16 minutes 52 seconds
Listed in: Education
Basics of File Systems
Published: Nov. 25, 2020, 9:15 a.m.Duration: 22 minutes 37 seconds
Listed in: Education
Cloud Computing
Published: Nov. 23, 2020, 10:30 a.m.Duration: 14 minutes 12 seconds
Listed in: Education
Scheduling
Published: Nov. 23, 2020, 10:10 a.m.Duration: 20 minutes 53 seconds
Listed in: Education
Virtualization
Published: Nov. 23, 2020, 9:50 a.m.Duration: 16 minutes 29 seconds
Listed in: Education
MR Algorithms
Published: Nov. 16, 2020, 10:25 a.m.Duration: 11 minutes 26 seconds
Listed in: Education
MapReduce Architecture
Published: Nov. 16, 2020, 10 a.m.Duration: 21 minutes 51 seconds
Listed in: Education
Sorting in Detail
Published: Nov. 16, 2020, 9:40 a.m.Duration: 16 minutes 25 seconds
Listed in: Education
Map - Sort - Reduce
Published: Nov. 16, 2020, 9:25 a.m.Duration: 14 minutes 47 seconds
Listed in: Education
Further Evolution
Published: Nov. 6, 2020, 10:20 a.m.Duration: 7 minutes 22 seconds
Listed in: Education
Open Source Big Data Stack
Published: Nov. 6, 2020, 10 a.m.Duration: 15 minutes 59 seconds
Listed in: Education
Google's Big Data Stack
Published: Nov. 6, 2020, 9:40 a.m.Duration: 19 minutes 42 seconds
Listed in: Education
The Big Data Stack
Published: Nov. 6, 2020, 9:30 a.m.Duration: 10 minutes 15 seconds
Listed in: Education
Introduction
Published: Nov. 3, 2020, 9:15 a.m.Duration: 1 hour 18 minutes 38 seconds
Listed in: Education