Introduction to Parallel Computing in Big Data Analysis (Part 1)

Running parallel and distributed AI tasks using Dask

Juan Nathaniel
Level Up Coding
Published in
6 min readMay 16, 2021

--

Artificial Intelligence (AI) tasks often involve large volumes of data from multiple sources of input. These datasets are often too large and too varied that a singular machine is unable to properly and efficiently process. Therefore, a cluster of machines is often sought after by parallelizing and batching these tasks.

--

--

Engineering @ Columbia University | Documenting and sharing my learning journey through AI, programming, and research