git and rsync folders

Posted on Tuesday, August 26, 2014

But, this article will be a little more focused and quick to the point.

I have a folder which has seven subfolders.  I do not want to track any file with git, but I do want to have easy access to rsync each folder to other computers.

Each individual folder will have its own rsync script that can retrieve its contents remotely.   I will use git to make a repository that only contains scripts and the seven folders.

Create the repository

> git init

Update .git/config  (not per say necessary)

> vi .git/config

Here is my config file

        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
        ignorecase = true
        packedGitLimit = 128m
        packedGitWindowSize = 128m

        deltaCacheSize = 128m
        packSizeLimit = 128m
        windowMemory = 128m

Create .gitignore file

For this git repository I only want to see my scripts, and the folders they are in.

> vi .gitignore

Place the following in it.

#Ignore Everything

# But not scripts or folders

This first ignores everything.  Then it adds two exceptions. Folders and anything ending in *.sh (my scripts)

Create Default script

Create a default script that will be copied to each directory.

> vi

And place the following in it (replacing the highlighted area with your own information)


#Remote location

#Check for an override name
if [ $2 ]

#loc is remote directory path


if [ "$1" == 'push' ]
  echo "Push it"
  #Need to make the directories
  #for folder
  ssh "$name"$url mkdir -p $loc${folder}
  rsync $flags * "$name"$url:$loc$folder
  echo "Pull it"
  rsync $flags "$name"$url:$loc$folder/* .

Create Copy Script

After this is done create a script that will copy this script into each folder, rename it and replace the XXXXX with is parent folder name.

> vi

And place the following in it.


for folder in `find $PWD -maxdepth 1 -mindepth 1 -type d -not -path "*/.git"`; do
  cp $folder/
  pushd .
    cd $folder
    basename $folder | xargs -I '{}' sed -i 's/XXXXX/{}/g'

This will find all folders with a maxdepth of 1, ignore the .git folder, and place the script in each folder.  It will then replace the XXXXX with the folder name.

Create Run All script

Create a run all script that will find all the scripts and run them.  (to make one easy push)

> vi

And place the following in it. (replace the highlighted with your username)


for folder in `find $PWD -iname "" -exec dirname {} \;`; do
  echo $folder
  pushd .
  cd $folder
  bash ./ push patman

Rsync all the data

After you have set up the base directory on the server you will rsync to run the script/

> ./

Commit git

> git add .
> git commit -m "initial commit"

Then on your git server init a bare repo that you can push your repository to.  In this example I am sudo'n to the git user (who does not have a normal shell, so you have to designate it)

> sudo su git -s /bin/bash
> cd /git/repos
> mkdir project.git
> cd project.git
> git --bare init

Finally push it up to a remote master server (adjust the url to your remote server and the path)

> git remote add origin
> git push origin master

Clone the repo and pull rsync

From another computer clone the repository

> git clone

After this head into one of the directories from the command line and pull your data (rsync) edit command for your username.

> ./ pull patman

I could have written on script to run all the scripts and pull (rsync) all the data down, but that is not my goal here.   My goal is to be able to download a folders contents, remotely, in a pinch.  Otherwise I do not want to take up the hard drive space (Or in my specific case my thumb drive's space)


No comments:

Post a Comment