Mimic is a set of tools to create mirrors of file collections. It uses a combination of techniques to optimize the synchronization of the collections. Mimic can use either the HTTP, SSH or FTP protocol to copy files. It supports both pushing files to remote locations or pulling files from remote locations. This means that the Mimic tools only needs to be installed on one of the systems in the set of mirror hosts.
Mimic was written to address the performance issues that exist in tools like bbcp, rsync and rcp. While these tools are widely used they are not optimized for synchronizing large data collections. For example, bbcp achieves high speed copying of files, but the process is blind to the state of the destination file. While bbcp does support the use of check sums using this feature slows down the copy process as the check sum is calculated on demand. With rsync only those files (blocks) which are different are copied, but the process of building the block maps occurs with each invocation. For a collection with thousands or millions of files the building of the block list is very time consuming. With rcp copies are completely blind with every file copied everytime. Mimic takes the best of what bbcp, rsync and rcp offer and combines the features into a single tool designed specifically to deliver and synchronize mirrors of file collections at multiple distributed locations.
The igpp-mimic library was developed using Java and will run on any platform with a supported Java Runtime Environment (JRE). The software was developed under Java version 1.7 and has only been tested with this version. You can determine which version of Java you are running with the command:
% java -version
java version "1.7.0_51"
OpenJDK Runtime Environment (rhel-2.4.4.1.el6_5-x86_64 u51-b02)
OpenJDK 64-Bit Server VM (build 24.45-b08, mixed mode)
Download the igpp-mimic package from the IGPP web site To use the jar file, first unbundle the distribution package, then place jar files in the "jar" folder in one of the folders on your Java search path. The package comes with convenience scripts in the "bin" folder which allows running the jar files without system wide installation.
For system wide installation the installation directory may vary from environment to environment but in UNIX-based environments it is typical to install software packages in the /usr/local directory and in Windows-based environments it is typical to install software packages in the C:\Program Files directory.
% jar -xvf igpp-mimic-1.1.9.jar
The unzip will result in the creation of the igpp-mimic-1.1.9 directory with the following directory structure:
A README file directing the user to the available documentation for the project.
The copyright notice from the Regents of the University of California detailing the restrictions regarding the use and distribution of this software. Although the license is strictly worded, the software has been classified as Technology and Software Publicly Available (TSPA) and is available for anyone to download and use.
This directory contains batch and shell scripts for executing the tool plus all necessary jar files.
This directory contains the jar file for this package.
This directory contains any necessary supporting jar files.
This directory contains documentation on the classes in the pds-cdf jar file. To view the documentation load the "index.html" file into a web browser.
Shell scripts to run the tool are in the "bin" directory in the distribution. You can run the tool on a Linux system with the command:
% ${package.command} task
% ${package.command}.bat task
Tasks that the Mimic tool can perform are:
Add any new files to the Mimic managed collection
Add any new files to the Mimic managed collection
Clone another Mimic managed collection
Configure synchronization with remote hosts
Determine information about the system and supported SSH protocol
Initialize a Mimic managed collection
Pull content from another Mimic managed archive
Push content to another Mimic managed archive
Refresh the Mimic files to reflect the current files in the folder
Count the number files in a branch of a file system
Report on the status of a Mimic managed collection