Migrate from local to remote

5 min readSep 6, 2020

Let’s find out how to use the command line terminal efficiently and reduce the distance between our remote work and office work 😎

Things covered: Cheatsheet for terminal users!!! 🤓

How to connect to our remote servers
How to run the programs remotely

Screen
nohup

How to run jupyter notebook remotely
How to share and save files between remote and local server
How to share files between two remote servers
How to create a virtual environment
Other Terminal shortcuts

How to connect to our remote servers

Connect to the VPN
Open the terminal and run

ssh -X remote_user_name@remote_ip

Give your password. (That’s it you are in !!!)
Navigate through the folders using

cd foldername

List out everything in the folder using command

ls

Run as a root user using

sudo su

with root permissions and give your password (After executing all the above steps)

How to run the programs remotely

Navigate to the folder in which you want to run your .py scripts using cd command and then run

python3 script_name.py

It can be any version (python/python2/python3)

What can be the main issues during running our programs remotely??

Due to VPN or the net connection, your running script might stop executing. So, for that, we can use screen commands and nohup commands to run scripts in the background.

Screen Command

Run in the terminal in which you are inside the remote server ( i.e. after ssh)

screen -S screen_name

screen_name can be anything, this command will create a screen on the remote server.

After the above command there will be a new screen, navigate to your folder where you want to run the command or the .py script

python3 script_name.py

Now, your program is executing in the new screen.
Press the below command to detach the screen

ctrl+a d

Now, you are out of your current screen and back to your remote path.
To resume and monitor the progress of your program, run

screen -r screen_name

To kill the current or any screen, press

screen -r screen_name
ctrl+a k

List out all the screen in the system using

screen -ls

Resume screen using the -r command and you can return back to your running script and monitor the outputs of the script. Before resuming the screen it’s important to detach the screen first.

nohup (No hang up)

To start a process in the background use the & symbol at the end of the command. In this example, we are running script_name.py and sending it to the background. Navigate to the folder in which you want to run your script using the cd command.

nohup python3 script_name.py &

The above command will make a default output log file as nohup.out in your working directory.
In case you want to change the name of output log file then use

nohup python3 script_name.py > custom_name.out &

To check the process when resuming the shell use the pgrep command as shown

pgrep -a python3

Check the progress of the script, using

tail -f nohup.out (-f for live session)ortail nohup.outorcat nohup.out

The tail command is used to print the last 10 lines of a file by default
The cat command displays the content of one or more text files on the screen without pausing.
kill PID (to kill the process)

kill process_id

To get the process id:

ps -ef (to find PID of your process)ps -ef | grep python3 (for only python3 process)

nohup command also returns the process id (PID)

How to run jupyter notebook remotely

Connect to VPN → ssh → Navigate to the folder you want to run jupyter notebook into using the cd command
Run this in remote terminal (In which you are connected through ssh)

jupyter notebook — no-browser — port=XXXX

Open another terminal in your local system, and un

ssh -N -f -L localhost:XXXX:localhost:XXXX remote_username@remote_iporssh -NL XXXX:localhost:XXXX remote_username@remote_ip

Lastly, open your browser and type

localhost:XXXX

XXXX port number can be anything like 8886 or 1223. In browser there will be a request asking for token so, copy the value of token from the remote terminal (see the above figure and copy the token= value)where you are running the using the jupyter notebook.

How to share and save files between remote and local server

Connect through VPN
Open your local terminal
For Remote-Local, run

(To copy a file, run the below command)

scp remote_username@remote_ip:remote_file_path local_file_save_path

Eg. remote_file_path=/home/my_files/data_words.txt

local_save_path=/User/username/Desktop/txt_data/

(To copy folder, run the below command)

scp -r remote_username@remote_ip:remote_file_path local_file_save_path

For Local-Remote, run

scp local_save_path remote_username@remote_ip:remote_file_path
(for file)scp -r local_save_path remote_username@remote_ip:remote_file_path
(for folder)

Eg. remote_file_path=/home/my_files/

local_save_path=/User/username/Desktop/txt_data/data_words.txt

How to share files between two remote servers

Login into both the servers using ssh.
Give permission to the folders/files for both servers using chmod command.

chmod 777 foldernameorchmod 777 . (if inside the folder)

Transfer the files using the below command in any of the ssh connected remote terminal.

scp remote_username1@ip1:from_file_path remote_username2@ip2:to_file_path(for file transfer)orscp -r remote_username1@ip1:from_folder_path remote_username2@ip2:to_folder_path(for folder transfer)

How to create a virtual environment:

pip install virtualenv

If you have a project in a directory called my-project you can set up virtualenv for that project by running:

cd my-project/
virtualenv venv

If you want your virtualenv to also inherit globally installed packages run

virtualenv venv — system-site-packages

You need to activate it first though (in every terminal instance where you are working on your project):

source venv/bin/activatepip install <package>It will get installed in the venv/ folder, and not conflict with other projects.To leave the virtual environment run:
deactivate

Other Terminal shortcuts:

To make a folder

mkdir folder_name

To delete a folder

rm -r folder_name

If the directory is not empty then the above command will ask to delete each file in the directory (folder_name)

To delete a directory (that is not empty) directly (only if you are sure that you want to delete everything in that folder)

rm -rf folder_name

To delete a file

rm file_name

To make a file

touch filenameor vim filename

eg. touch python1.py or data.txt, vim python1.py

To move a file

mv old_file_name  new_file_name

To move a data.txt file from /home/Desktop/ to /home/Desktop/all_data/

old_file_name=’/home/Desktop/data.txt’

new_file_name=’home/Desktop/all_data/new_name.txt’ (for name other than data.txt)

or new_file_name=’home/Desktop/all_data/data.txt’ (for same name)

Copy a file

cp -r source_path destination_path

Download zip file from web to remote server

Right Click on the zip file and copy link address
Open terminal and navigate to the folder where you want to save this file run

wget copied_link

3. To unzip the file, run

unzip filename

To zip a file

zip file_name

Migrate from local to remote

Screen Command

nohup (No hang up)

Written by Swatimeena