This talk is part of the Pangeo Show & Tell series and was given on September 1st 2022 by Felix Cremer. Bio Felix Cremer received his diploma in mathematics from the University of Leipzig in 2014. In 2016 he started his PhD study on time series analysis of hypertemporal Sentinel-1 radar data. He currently works at the Max-Planck-Institute for Biogeochemistry on the development of the JuliaDataCubes ecosystem in the scope of the NFDI4Earth 5 project. Abstract The Earth Data Lab (EDL) is a data cube framework in Julia for the efficient handling of raster data. It is based on the YAXArrays.jl package. YAXArrays.jl provides functionality to deal with labelled arrays, similar to the xarray python package and it also provides efficient and easy multithreading and distributed computation of user defined functions along arbitrary slices of the data. EarthDataLab.jl uses DiskArrays.jl in the backend to deal with out of memory datasets. In this Show-and-Tell Felix is going to give a short introduction into the EarthDataLab.jl package for raster data handling in Julia.
Handling large geo data with Julia - snapshot
In this Show-and-Tell Felix is going to give a short introduction into the EarthDataLab.jl package for raster data handling in Julia.
The Earth Data Lab (EDL) is a data cube framework in Julia for the efficient handling of raster data.
This talk is part of the Pangeo Show & Tell series and was given on September 1st 2022 by Felix Cremer.
Felix Cremer, and Pangeo Europe. "Handling large geo data with Julia." ROHub. Sep 02 ,2022. https://doi.org/10.24424/2byf-7r07.
This is the recorded talk from Felix Cremer during the Pangeo Show & Tell in September 1st, 2022. Felix is going through his Julia Notebook and explain us about handling large geo data with Julia.
Youtube video "Handling large geo data with julia by Felix Cremer."
This is the shared document we use for all the Pangeo Show and Tell. We collect information, Q&A and feedback. Each Show and Tell has its own sub-section.
Jupyter Notebook used by Felix during the Pangeo Show & Tell to demonstrate how to use EarthDataLab.jl to do large scale computations. To execute this Jupyter Notebook, data contained in the "input folder" is needed (please create a folder called "data" in the folder where you have stored the notebook).
How to use EarthDataLab.jl to do large scale computations (Jupyter Notebook)
Part of ne_50m_admin_0_countries shapefile (projection information).
You will find here all the information published to advertise the Pangeo Show & Tell Talk frm Felix Cremer on "Handling large geo data with Julia ".
Pangeo discourse post announcing 1st September Show & Tell by Felix Cremer.
Part of ne_50m_admin_0_countries shapefile.
YAXArrays.jl is another xarray-like Julia package. A package for operating on out-of-core labeled arrays, based on stores like NetCDF, Zarr or GDAL. Package Features: - open datasets from a variety of sources (NetCDF, Zarr, ArchGDAL) - interoperability with other named axis packages through YAXArrayBase - efficient mapslices(x) operations on huge multiple arrays, optimized for high-latency data access (object storage, compressed datasets)
This will become a selection of tutorials on the use of ESDL.jl and YAXArrays.jl julia packages for the handling of large scale out-of-core geospatial datasets.