High Performance Python - Gus Cavanaugh | PyData Global 2021

YouTube Excerpt: High Performance Python With Numba, Dask, and Rapids For the Absolute Beginner Speaker: Gus Cavanaugh Summary Data Scientists often have large datasets and powerful hardware at their disposal. However, the excitement of fast computation in Python slows against a steep learning curve. This talk will build your confidence and intuition around high performance computing with Python. We step through a complete example while also covering the core concepts so you can generalize to your own work. Description An example data science pipeline with numpy and pandas Common heuristics for when to accelerate your code Quick survey of common approaches An example data processing pipeline with numpy How to accelerate on a single machine with Numba Brief introduction to Numba Quick comparison to cython Accelerating our example pipeline with numba How to distribute on a cluster with Numba and Dask Brief introduction to Dask Quick comparison to PySpark, Ray Accelerating our example pipeline with numba and dask How to accelerate and distribute with Numba, Dask, and Rapids Brief introduction to Rapids & GPUs Quick comparison to other GPU computing methods Accelerating our example pipeline with numba, dask, and rapids Conclusion Review of performance gains Summary of when to apply each to your project Where to find hardware and example costs for various pipelines and data volumes Gus Cavanaugh's Bio As a former consultant, I worked on data projects big and small for organizations like Booz Allen Hamilton and IBM GBS. Like many others, I discovered Python as a wonderful swiss army knife and used it on client projects for tasks ranging from wrangling data into spreadsheets, building quick APIs, and processing data over clusters of machines. Four years ago, I left consulting to work on the software vendor side, moving to Anaconda. Today, I work at Coiled, founded by the creator of Dask, Matt Rocklin. My family and I reside in Winston-Salem, NC. GitHub: https://github.com/gcav66/ Twitter: https://twitter.com/GusCavanaugh/ LinkedIn: https://www.linkedin.com/in/gustafrcavanaugh/ PyData Global 2021 Website: https://pydata.org/global2021/ LinkedIn: https://www.linkedin.com/company/pydata-global Twitter: https://twitter.com/PyData www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R. PyData conferences aim to be accessible and community-driven, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases. 00:00 Welcome 01:15 Faking a Scientific Workload 02:35 Working with Numba 05:19 About Gus Cavanaugh 07:19 About Rapids. Why do we need it? 12:27 What is Dask? 15:35 Using Dask 20:00 Summarizing everything up to now 23:07 Setting up Rapids 26:00 Running Numba in Dask 28:50 Ways to deploy Dask cluster 29:55 Cost of setting up a Rapids client and Dask cluster 31:05 Running everything in the local machine: The Afar package S/o to https://github.com/Retinpkumar for the video timestamps! Want to help add timestamps to our YouTube videos to help with discoverability? Find out more here: https://github.com/numfocus/YouTubeVideoTimestamps

High Performance Python With Numba, Dask, and Rapids For the Absolute Beginner Speaker: Gus Cavanaugh Summary Data Scientists often have large...

Read Full Article 🔍

Curious about High Performance Python - Gus Cavanaugh | PyData Global 2021's Color? Explore detailed estimates, income sources, and financial insights that reveal the true scope of their profile.

Source ID: ewaY9CcjLt0

Category: color style guide