xbatcher for Machine Learning (Part 1) Cookbook
This Project Pythia Cookbook covers a complete workflow for a convolutional neural network. Here, we emphasize how to create xarray-based training datasets with the
By the end of this tutorial, you should be able to use core features of
xbatcher to create valid training datasets for a convolutional neural network. You should also be able to recombine the CNN results into a valid xarray dataset for further viewing and analysis. Additionally, this tutorial shows some software designs that could be useful for organizing your ML experiments in the future.
This Cookbook is mostly broken up by stages of the ML workflow, and is designed such that the training, testing, and prediction phases can be run somewhat separately. While every stage of this workflow is included explicitly, the training, testing, and prediction workflows are also included as separate functions that can be run instead of the inline sections. This enables you to minimize or remove sections you have already run, or already understand.
Running the Notebooks
You can either run the notebook using Binder or on your local machine.
Running on Binder
The simplest way to interact with a Jupyter Notebook is through Binder, which enables the execution of a Jupyter Book in the cloud. The details of how this works are not important for now. All you need to know is how to launch a Pythia Cookbooks chapter via Binder. Simply navigate your mouse to the top right corner of the book chapter you are viewing and click on the rocket ship icon, (see figure below), and be sure to select “launch Binder”. After a moment you should be presented with a notebook that you can interact with. I.e. you’ll be able to execute and even change the example programs. You’ll see that the code cells have no output at first, until you execute them by pressing Shift+Enter. Complete details on how to interact with a live Jupyter notebook are described in Getting Started with Jupyter.
Running on Your Own Machine
If you are interested in running this material locally on your computer, you will need to follow this workflow:
(Replace “cookbook-example” with the title of your cookbooks)
git clone https://github.com/ProjectPythia/cookbook-example.git
Move into the
Create and activate your conda environment from the
conda env create -f environment.yml conda activate cookbook-example
Move into the
notebooksdirectory and start up Jupyterlab
cd notebooks/ jupyter lab