Run Jupyter on Remote Server
Hi all! Jupyter Notebook is easy to use the platform to write python codes. However, sometimes we might need to work on remote servers for high CPU/GPU or other reasons. In this blog post, you can learn that how can we use the Jupyter platform on a remote server with the virtual environment. Thus, you can download files from the server to your local, you can connect to .ipynb files, you can see the content of .csv,.txt .etc files, you can use the CPU/GPU of the server.
- Installation of Jupyter
Firstly, we need to install jupyter in our server account. You can use pip for this step:
pip install jupyter
2. Virtual Environment
Virtual environment is essential for all programmers to avoid package conflicts. Separate projects require different libraries and their versions might vary. Therefore, it is the best way to run each project in its own environment. We can set up an environment via virtualenv package. Let’s install it by pip.
pip install virtualenv
Now, you can create a virtual environment with a valid path. My suggestion is that, make a directory for your all environments and set up them in it. For instance, we have a directory ‘venvs’ and we will build a virtual environment ‘env-medium’ in this directory. We will use this command:
source venvs/env-medium/bin/activate
After that step, you can see the name of the environment to the left of your user name. Right now you can install all necessary libraries to use in the project!
If you want to exit from the environment just write it on the terminal:
deactivate
After this command, you can see that the environment name will disappear from the left of your user name.
3. Run Jupyter on Server
3.1 Server-Side
Jupyter runs each notebook on kernels. Because of that reason, if we install a kernel to the jupyter with the activated virtual environment, we can connect or create notebooks with this specific kernel. Thus, we can use all libraries in a virtual environment that is activated when we create a kernel. Now, let’s install our kernel step by step.
3.1.1 Install Kernel
In this step, the most important part is that, don’t forget to activate your desired virtual environment before the installation of the kernel to the jupyter. Because we would like to use a specific environment and its libraries on jupyter project. So, let's activate our virtual environment:
source venvs/env-medium/bin/activate
After activating our environment, we will install ipykernel package to avoid possible kernel conflict between the local and server sides.
pip install ipykernel
Now, we can create our kernel with the desired name and install it on jupyter. Let’s create a kernel “deneme-medium”.
python -m ipykernel install — user — name=”deneme-medium”
By installing our kernel to jupyter, we can use all libraries in env-medium via ‘deneme-medium’ kernel in Jupyter Notebook. After that step, we can deactivate our environment to open a port in the remote server (I will use 8892). This port will send the necessary information to our local port.
jupyter notebook — no-browser — port=8892
P.S: our kernel ‘deneme-medium’ will always remain defined to your jupyter path and you will not need to do these steps next time. You just need to open a port for each time. Of course, if you want to install a new library, activate the environment that you activated when installing your kernel, and install via pip.
3.2 Local-Side
We have a port that is ready to send information from the server. So let's listen to this port from our local computer. You can use the following command:
ssh -L 8889:localhost:8892 username@server_adress
- -L: open a local port
This command-line opens port 8889 in your local computer and start to listen to port 8892.
Until here we did all back-end jobs. Then, let’s open your web browser and write localhost:8889.
After that, you will see a jupyter gate that will ask for a token for entry. We will take this entry token from the server side. The image below will explain it much more clearly.
Voila! You can now use all jupyter features on the server :)
You can select your kernel name when you create a new notebook, by using ‘New’ button. If you need a new library just connect to the server with another terminal and activate your environment. After installing your desired library just restart your kernel in Jupyter platform and your library is ready to use :)
I hope this blog post can help you to find higher CPU/GPU devices and using all features of jupyter on the server :)
