Migrate from local to remote

5 min readSep 6, 2020
Photo by Daan Stevens on Unsplash

Let’s find out how to use the command line terminal efficiently and reduce the distance between our remote work and office work 😎

Things covered: Cheatsheet for terminal users!!! 🤓

How to connect to our remote servers

How to run the programs remotely

How to run jupyter notebook remotely

How to share and save files between remote and local server

How to share files between two remote servers

How to create a virtual environment

Other Terminal shortcuts

How to connect to our remote servers

  • Connect to the VPN
  • Open the terminal and run
ssh -X remote_user_name@remote_ip
  • Give your password. (That’s it you are in !!!)
  • Navigate through the folders using
cd foldername
  • List out everything in the folder using command
  • Run as a root user using
sudo su

with root permissions and give your password (After executing all the above steps)

How to run the programs remotely

  • Navigate to the folder in which you want to run your .py scripts using cd command and then run
python3 script_name.py 

It can be any version (python/python2/python3)

  • What can be the main issues during running our programs remotely??

Due to VPN or the net connection, your running script might stop executing. So, for that, we can use screen commands and nohup commands to run scripts in the background.

Screen Command

  • Run in the terminal in which you are inside the remote server ( i.e. after ssh)
screen -S screen_name

screen_name can be anything, this command will create a screen on the remote server.

  • After the above command there will be a new screen, navigate to your folder where you want to run the command or the .py script
python3 script_name.py
  • Now, your program is executing in the new screen.
  • Press the below command to detach the screen
ctrl+a d
  • Now, you are out of your current screen and back to your remote path.
  • To resume and monitor the progress of your program, run
screen -r screen_name
  • To kill the current or any screen, press
screen -r screen_name
ctrl+a k
  • List out all the screen in the system using
screen -ls

Resume screen using the -r command and you can return back to your running script and monitor the outputs of the script. Before resuming the screen it’s important to detach the screen first.

nohup (No hang up)

  • To start a process in the background use the & symbol at the end of the command. In this example, we are running script_name.py and sending it to the background. Navigate to the folder in which you want to run your script using the cd command.
nohup python3 script_name.py &
  • The above command will make a default output log file as nohup.out in your working directory.
  • In case you want to change the name of output log file then use
nohup python3 script_name.py > custom_name.out &
  • To check the process when resuming the shell use the pgrep command as shown
pgrep -a python3
  • Check the progress of the script, using
tail -f nohup.out (-f for live session)ortail nohup.outorcat nohup.out
  • The tail command is used to print the last 10 lines of a file by default
  • The cat command displays the content of one or more text files on the screen without pausing.
  • kill PID (to kill the process)
kill process_id
  • To get the process id:
ps -ef (to find PID of your process)ps -ef | grep python3 (for only python3 process)
  • nohup command also returns the process id (PID)

How to run jupyter notebook remotely

  • Connect to VPN → ssh → Navigate to the folder you want to run jupyter notebook into using the cd command
  • Run this in remote terminal (In which you are connected through ssh)
jupyter notebook — no-browser — port=XXXX
  • Open another terminal in your local system, and un
ssh -N -f -L localhost:XXXX:localhost:XXXX remote_username@remote_iporssh -NL XXXX:localhost:XXXX remote_username@remote_ip
  • Lastly, open your browser and type
Running jupyter notebook

XXXX port number can be anything like 8886 or 1223. In browser there will be a request asking for token so, copy the value of token from the remote terminal (see the above figure and copy the token= value)where you are running the using the jupyter notebook.

How to share and save files between remote and local server

  • Connect through VPN
  • Open your local terminal
  • For Remote-Local, run

(To copy a file, run the below command)

scp remote_username@remote_ip:remote_file_path local_file_save_path

Eg. remote_file_path=/home/my_files/data_words.txt


(To copy folder, run the below command)

scp -r remote_username@remote_ip:remote_file_path local_file_save_path
  • For Local-Remote, run
scp local_save_path remote_username@remote_ip:remote_file_path
(for file)
scp -r local_save_path remote_username@remote_ip:remote_file_path
(for folder)

Eg. remote_file_path=/home/my_files/


How to share files between two remote servers

  • Login into both the servers using ssh.
  • Give permission to the folders/files for both servers using chmod command.
chmod 777 foldernameorchmod 777 . (if inside the folder)
  • Transfer the files using the below command in any of the ssh connected remote terminal.
scp remote_username1@ip1:from_file_path remote_username2@ip2:to_file_path(for file transfer)orscp -r remote_username1@ip1:from_folder_path remote_username2@ip2:to_folder_path(for folder transfer)

How to create a virtual environment:

  • Run
pip install virtualenv

If you have a project in a directory called my-project you can set up virtualenv for that project by running:

cd my-project/
virtualenv venv

If you want your virtualenv to also inherit globally installed packages run

virtualenv venv — system-site-packages

You need to activate it first though (in every terminal instance where you are working on your project):

source venv/bin/activatepip install <package>It will get installed in the venv/ folder, and not conflict with other projects.To leave the virtual environment run:

Other Terminal shortcuts:

  • To make a folder
mkdir folder_name
  • To delete a folder
rm -r folder_name

If the directory is not empty then the above command will ask to delete each file in the directory (folder_name)

  • To delete a directory (that is not empty) directly (only if you are sure that you want to delete everything in that folder)
rm -rf folder_name
  • To delete a file
rm file_name
  • To make a file
touch filenameor vim filename

eg. touch python1.py or data.txt, vim python1.py

  • To move a file
mv old_file_name  new_file_name

To move a data.txt file from /home/Desktop/ to /home/Desktop/all_data/


new_file_name=’home/Desktop/all_data/new_name.txt’ (for name other than data.txt)

or new_file_name=’home/Desktop/all_data/data.txt’ (for same name)

  • Copy a file
cp -r source_path destination_path
  • Download zip file from web to remote server
  1. Right Click on the zip file and copy link address
  2. Open terminal and navigate to the folder where you want to save this file run
wget copied_link

3. To unzip the file, run

unzip filename
  • To zip a file
zip file_name




Data Scientist @Sprinklr | IIT Bombay | IIT (ISM) Dhanbad