User Rating: 5 / 5

Star ActiveStar ActiveStar ActiveStar ActiveStar Active
 
Since the time of writing of this article unison has added a native changes check without inotify. Thought I still use the method explained in the article, you may give a try. Check stackoverflow

spider.24990811 stdGiven

Linux remote server and Linux (OpenSuse) local machine. SSH access to the remote server with sudo (optional) permissions.

Task

I develop a web-site deployed at a remote server. I have a to-do list and as a rule I backup the web-site, place it at my local comp, work, and place it back to the production server when finished. But what if the client wants to work at the web-site at the same time? E.g. to work with the conent? Using FTP at my side is not an option as it takes not only time, but doesn't allow e.g. to search text in many files using e.g. Krusader. Besides, I cannot use my favourite Geany editor when using KDE built-in FTP or SFTP (fish) protocols. I mean I cannot edit a file from Krusader FTP/SFTP connection in Geany, only in KWrite.

So I want to work at the remote web-site as if I work at the local one. As if the remote web-site folder is mounted like a normal hard disk. But all the ftp/ssh mounts I managed to find were slow, incredible slow.

Theory

There is an awesome Unison tool which allows to synchronize two folders - local and remote (or remote and remote, or local and local etc). But the minus is that after any file change you have to run Unison manually to replicate the file changes. So we need to make Unison automatically run each time we change anything in the web-site folder.

So here are the steps in general we must perform:

  1. Make password-less SSH login to the remote server.
  2. Install Unison at both local and remote machines.
  3. Synchronize the folders for the first time manually.
  4. Make a script using inotifywait to watch changes at the local host and run Unison each time something is changed.

Solution

Make password-less SSH login to the remote server

Open terminal at your local machine and run command:

ssh-copy-id -i root@yoursite.com.ua

You'll be asked to accept the remote server RSA key fingerprint and then you'll be requested to enter the root password.

Here is my output:

gruz@gruz:/mnt/sda2/home/gruz>;; ssh-copy-id -i root@yoursite.com.ua
The authenticity of host 'yoursite.com.ua (95.95.95.95)' can't be established.
RSA key fingerprint is 48:36:22:32:45:09:1d:04:eb:63:d1:60:27:7e:f6:ba.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'yoursite.com.ua' (RSA) to the list of known hosts.
root@yoursite.com.ua's password: 
Now try logging into the machine, with "ssh 'root@yoursite.com.ua'", and check in:
  ~/.ssh/authorized_keys
to make sure we haven't added extra keys that you weren't expecting.
gruz@gruz:/mnt/sda2/home/gruz>;

Image-missing.svg

Now you must be able to login without password request:

gruz@gruz:/mnt/sda2/home/gruz>;ssh root@yoursite.com.ua
Last login: Tue Aug 28 12:34:13 2012 from 174-235-133-95.pool.ukrtel.net
[root@portall ~]#

Image-missing.svg

This must be done only once.

Install Unison at both local and remote machines

At my local OpenSuse I do:

sudo zypper in unison

At my remote CentOS I do

yum install unison

Note: You must have compatibe Unison versions (as a rule the same) to be installed at both machines. But modern Linux OSes have the same ones by default, so don't care.

Synchronize the folders for the first time manually

Optional. For the first time it's worth to manually copy the web-site to your local folder. This may be much quicker, then any other way. So I backup my Joomla web-site with AkeebaBackup and extract it to a local folder.

Anyway you have to know local absolute path and remote absolute path.

So my local path is /home/gruz/www/bulat.uaa/www_sync and my remote path is /some/remote/path/bulat.ua

Run Unison, it will open a GUI window.

Image-missing.svg

Press Add and follow the steps:

Image-missing.svg Image-missing.svg
Image-missing.svg Image-missing.svg
Image-missing.svg Image-missing.svg

Next you'll see a window with proposed actions. Some are "not sure". Select all and click "Right to Left"

Image-missing.svg

Press go and wait. When it's done you may close the window.

Image-missing.svg

I recommend to open Unison GUI and make manual synchronization each time before you start working with your web-site so easily see and resolve (if needed) "not sure" tasks. As a rule you must set all "Right to Left".

Make a script using util inotifywait to watch changes at the local host and run Unison each time something is changed

Ok. Now you can edit a file at your local host, save it and run unison from terminal:

unison -ui text -auto -batch bulat.ua

Remember? bulat.ua is my profile name.

So in my local folder I edit README.txt file and run the comman above:

Output is:

gruz@gruz:/mnt/sda2/home/gruz>;; unison -ui text -auto -batch bulat.ua
Contacting server...
Connected [//gruz//mnt/sda2/home/gruz/work/www/bulat.uaa/www_sync -> //some/remote/path/bulat.ua]
Looking for changes
  Waiting for changes from server
Reconciling changes
changed  ---->            README.txt  
Propagating updates
UNISON 2.40.63 started propagating changes at 13:58:09.03 on 28 Aug 2012
[BGN] Updating file README.txt from /mnt/sda2/home/gruz/work/www/bulat.uaa/www_sync to //some/remote/path/bulat.ua
[END] Updating file README.txt
UNISON 2.40.63 finished propagating changes at 13:58:09.47 on 28 Aug 2012
Saving synchronizer state
Synchronization complete at 13:58:10  (1 item transferred, 0 skipped, 0 failed)
gruz@gruz:/mnt/sda2/home/gruz>;

Image-missing.svg

How to make it run automatically at every file change?

You must have inotifywait tool installed at your local machine. In OpenSuse it's called inotify-tools and can be found here http://software.opensuse.org/package/inotify-tools .

So my approach to use it is to go to one level upper then my local folder. This means in my case /home/gruz/www/bulat.uaa and create a file sync.sh there with the following content:

#!/bin/sh
while inotifywait -r -e modify -e create -e delete ~/www/bulat.uaa/www_sync/; do
    unison -ui text -auto -batch bulat.ua
done

Image-missing.svg

Make the file executable.

cd //home/gruz/www/bulat.uaa; chmod +x sync.sh

And run the file in terminal. Till terminal is open the synchronization works. When you stop working, close the terminal window or Ctrl+C.

gruz@gruz:~/www/bulat.uaa>;; ./sync.sh
Setting up watches.  Beware: since -r was given, this may take a while!
Watches established.

Image-missing.svg

Now if I edit and change README.txt file I can see such an output in the terminal:

gruz@gruz:~/www/bulat.uaa>; ./sync.sh
Setting up watches.  Beware: since -r was given, this may take a while!
Watches established.
/home/gruz/www/bulat.uaa/www_sync/ CREATE .goutputstream-Q5DJJW
Contacting server...
Connected [//gruz//mnt/sda2/home/gruz/work/www/bulat.uaa/www_sync -> //some/remote/path/bulat.ua]
Looking for changes0
  Waiting for changes from server
Reconciling changes
changed  ---->            README.txt  
Propagating updates
UNISON 2.40.63 started propagating changes at 14:16:12.02 on 28 Aug 2012
[BGN] Updating file README.txt from /mnt/sda2/home/gruz/work/www/bulat.uaa/www_sync to //some/remote/path/bulat.ua
[END] Updating file README.txt
UNISON 2.40.63 finished propagating changes at 14:16:12.64 on 28 Aug 2012
Saving synchronizer state
Synchronization complete at 14:16:13  (1 item transferred, 0 skipped, 0 failed)
Setting up watches.  Beware: since -r was given, this may take a while!
Watches established.

Image-missing.svg

Next I recommend to make the terminal window of the smallest possible size and place it on top of other windows in a corner to see the replication process.

Image-missing.svgImage-missing.svg

Limitations

  • The replication is not light-speed. It takes as a rule a part of a second. That's why you have to see the on-top terminal window to be informed when it's done. Ususally it takes the time you press alt+tab to switch from your editor to your browser.
  • After some time you don't work with the remote web-site there can appear files (cache, images, other admins edit CSS files) which Unison cannot resolve it'self. Then it asks the user what to do. That's why before you start working with the web-site run Unison GUI, do synchronization answering the "not sure" questions , close it and then run your script in the terminal.
  • Since the synchronization takes some time the inotify/unison pair can ignore a change if you very quckily save a file till another file is replicating. So some minimal attention is needed.

Other

You may need 1-directional synchronization, when you don't care about the changed someone else can do at the server. In this case a better and quicker and easier tool is RealSync: replicate developer's files over SSH in realtime

Comments   

Kitesurfer
+1 # Kitesurfer 2016-05-13 21:45
Hi
Even a few years later, still very helpful and well done.
Many thanks for that
gruz
# gruz 2016-05-13 22:22
Quoting Kitesurfer:
Hi
Even a few years later, still very helpful and well done.
Many thanks for that


You may check also http://stackoverflow.com/questions/16496624/gaining-better-performance-with-inotify-tools-and-unison
I didn't try it yet.

Please register to post comments.

Found a typo? Please select it and press Ctrl + Enter.
FaLang translation system by Faboba
ОУН-УПА: легенда спротиву Українська Дивізія «Галичина» Зиновій Книш: спогади й матеріали до діяння УВО і ОУН Реаніматор електронних текстів Забуттю не підлягає 1917-1991

See a typo?

Select the text block and press

CTRL+Enter

Comments

  • NotificationAry - get emails on Joomla content is submitted, added or changed

    Berndi Berndi 30.06.2019 11:45
    The fields %TO_NAME% and %TO_USERNAME% are empty

    Read more...

     
  • NotificationAry - get emails on Joomla content is submitted, added or changed

    Berndi Berndi 30.06.2019 11:43
    My fault, I made it

    Read more...

     
  • NotificationAry - get emails on Joomla content is submitted, added or changed

    Berndi Berndi 30.06.2019 10:46
    I did. Now I'm asking me how to put the surename into the email... :-|

    Read more...

     
  • NavigationAry - navigate between Joomla menu items like between articles using "prev - next"

    Berndi Berndi 29.06.2019 15:59
    I also miss content in %TO_NAME%

    Read more...

     
  • NotificationAry - get emails on Joomla content is submitted, added or changed

    Berndi Berndi 29.06.2019 15:15
    Hi gruz, I have the problem, that when I use the %AUTHOR% tag, the plugin dosen't send the author ...

    Read more...