Introduction to Jupyter Notebooks
The Jupyter Notebook is an interactive web application that allows viewing, creation and documentation of live code.Notebook applications include data transformation and visualisation.
- What is Jupyter?
- This is a Title in Markdown
As mentioned in the instructions, all materials can be open in Colab as Jupyter notebooks. In this way users can run the code in the cloud. It is highly recommanded to follow the tutorials in the right order.
What is Jupyter?
The Jupyter Project is an open source effort that evolved from the IPython project to support interactive data science and computing. Besides Python, it also supports many different programming languages including R and Julia.
Components of Jupyter Notebook
-
Jupyter Notebook IDE: The application that launches in a web browser like Firefox or Safari and is the environment where you write and run your code.
-
Jupyter Notebook Files(.ipynb): The file format that you can use to store code and markdown text for individual projects and workflows.
-
Kernels: A kernel runs your code in a specific programming language. In this tutorial, Python kernel is used within the Jupyter Notebook IDE.
Jupyter Notebook User Interface
After you create a new notebook file (.ipynb), you will be presented with notebook name, menu bar, tool bar and a code cell as default starting cell.
-
Notebook Name: if you click at the notebook name, you could rename the file.
-
Menu Bar: presents all functions and settings of the notebook file.
-
Tool Bar: presents the most used tools as icons.
-
Code Cell: it is the default type of cell when you create a new cell; if you want to transfer it to a markdown cell, you could use the drop down box in tool bar or a keyboard shortcut.
1. Headers
Note: The more # the title holds, the smaller the title will be.
# This is a Title in Markdown
## This is a Subtitle in Markdown
### This is a smaller subtitle in Markdown
will be rendered as:
This is a Title in Markdown
This is a Subtitle in Markdown
This is a smaller subtitle in Markdown
2. Lists
* This is a bullet list
* This is a bullet list
* This is a bullet list
1. This is a numbered list
2. This is a numbered list
3. This is a numbered list
will be rendered as:
- This is a bullet list
- This is a bullet list
- This is a bullet list
- This is a numbered list
- This is a numbered list
- This is a numbered list
Here is some highlighted text!
6. Hyperlinks
You can use HTML in Markdown cells to create hyperlinks redirecting to other websites. For example, the following syntax.
More infos about our data cube program can be found at <a href="https://datacube.remote-sensing.org/">this website</a>
will be rendered as:
More infos about our data cube program can be found at this website
Or if the image need to be resize:
<div>
<img src=https://i.imgur.com/VGPeJ6s.jpg width="200">
</div>
will be rendered as:
Note: The texts in the rectangle brackets (e.g. "Fotograph of Philip is here") will appear when the image fails to load.
8. LaTex
Jupyter notebook markdown cell also supports LaTex. So that the markdown cells interpret your texts as LaTex, surround your input texts with $ signs.
For instance, $c=a+b$ will be rendered as
$c=a+b$
If you want your texts be centered in the cell, surround your input texts with two $ signs.
For instance, $$C_{g}=\frac{H}{\frac{\pi}{2}*Cl_{p}}$$ will be rendered as
$$C_{g}=\frac{H}{\frac{\pi}{2}*Cl_{p}}$$
and
|Uppercase| LaTeX |Lowercase| LaTeX |
|---------|-------|---------|-------|
|$\Delta$ |\\Delta|$\delta$ |\\delta|
|$\Omega$ |\\Omega|$\omega$ |\\omega|
will be rendered as:
| Uppercase | LaTeX | Lowercase | LaTeX |
|---|---|---|---|
| $\Delta$ | \Delta | $\delta$ | \delta |
| $\Omega$ | \Omega | $\omega$ | \omega |
9. Table
A table can be constructed using | (pipe symbol) and — (dash) to mark columns and rows. The first row of the table defines the headers, and the next row defines the alignment of each column.
For instance,
| Stretch/Untouched | ProbDistribution | Accuracy |
| --- | --- | --- |
| Stretched | Gaussian | .843 |
will be rendered as:
| Stretch/Untouched | ProbDistribution | Accuracy |
|---|---|---|
| Stretched | Gaussian | .843 |
The widths of the columns can also be changed by adding an empty row at the end with defined width.
| Stretch/Untouched | ProbDistribution | Accuracy |
| --- | --- | --- |
| Stretched | Gaussian | .843 |
|<img width=200/>|<img width=200/>|<img width=200/>|
will be rendered as:
| Stretch/Untouched | ProbDistribution | Accuracy |
|---|---|---|
| Stretched | Gaussian | .843 |
What is Colab?
So now we learnt about Jupyter Notebook, then what is Colab? Colab notebooks are Jupyter notebooks that are hosted by Colab. Therefore, the operations are highly similar and it allows you to write and execute Python in your browser, with
- Zero configuration required
- Free access to GPUs
- Easy sharing
There are limits in the available memory but they are generally sufficient if you are not performing particularly demanding tasks. One of the benefits using Colab is that you do not need to worried about dependencies as much for downloading Python tools and libraries. Many of the libraries are available in the Colab environment by default, eg. Pandas.
To import a library that's not in Colaboratory by default, you can also use !pip install or !apt-get install.
For example,
!pip install cartopy
import cartopy
Also, because Colab is a cloud environment, you cannot directly assess files in your local environment. If you want to import files, there are two options:
Data Import
Option 1: Upload Files
Remarks: The file name in line pd.read_csv() need to be exactly the same as the name of the file you uploaded to not run into errors.
from google.colab import files # use the google.colab library
uploaded = files.upload() # upload it, click choose file after running this cell and pick the file
# read your file depends on the file format
# Let's say you have a csv, then you can read them using Pandas (you will learn more about Pandas later)
import io
import pandas as pd
df = pd.read_csv(io.BytesIO(uploaded['yourfile.csv']))
df # now your data is stored in this data frame
Option 2: Access Files from Google Drive
One of the cons using upload option is that you can only upload one file at a time and it might take sometimes if your files are large. Let's say your colleague share you a file from Google Drive and you have a copy of it in the Drive. It is probably not the best way to download them to your computer and upload them in Colab again.
What you can do is to directly use files on the drive.
After running the following code, you still need to sign in to your Google Account, and click Allow to permit access to your own Google Drive.
from google.colab import drive
drive.mount('/content/drive/')
After successful connection, you can see "Mounted at / content/drive/" under the cell.
Then, you can check your directory.
!ls
Where to search for your files depends on where do you store them. Let's say you stored your file inside Google Drive with a folder call yourfolder. You can check if the files are there by:
Remark: Be carefule there is no space between My and Drive
print(os.popen('ls /content/drive/MyDrive/yourfolder').read())
Your files should appear under the cell after run.
Then you can open your file using the path. You can specify your path in a raw string r"path".
Then you can open them using different methods depending on the data format. The following example is a Geotiff file and it can be opened using gdal (You do not need to know much about it for now).
path = r"/content/drive/MyDrive/yourfolder/boundary.tif"
# import library
import gdal
# open file
ds = gdal.Open(path, gdal.GA_ReadOnly)
# import library
import pandas as pd
# define your path
path = r"/content/drive/MyDrive/yourfolder/city.csv"
# open file
df = pd.read_csv(path)
df # now your data is stored in this data frame
from google.colab import files # import library; only need to be done once for the whole notebook
df.to_csv('temple_location.csv', encoding='utf_8_sig', index=False) # suppose df is the dataframe you want to export
files.download('temple_location.csv') # run this code and the file will be directly downloaded
What you can also do it to run only:
from google.colab import files # import library; only need to be done once for the whole notebook
df.to_csv('temple_location.csv', encoding='utf_8_sig', index=False) # suppose df is the dataframe you want to export
# by default they will be in /content
and then click the bar on the left side with the folder icon (Files). Then you can also search for your files in the directory and click on the three dots and then click download.
It is generally recommanded to modify a file already existed in the Google Drive but not to create new files in Google Drive.
Previous Lesson: Python Research Q&A Basics
Next Lesson: Python Introduction Basics
Additional information
This notebook is provided for educational purpose and feel free to report any issue on GitHub.
Author: Ka Hei, Chow
License: The code in this notebook is licensed under the Creative Commons by Attribution 4.0 license.
Last modified: December 2021