Name

gfpcopy — copy Gfarm files in parallel

Synopsis

gfpcopy [-nqvdpPU] [-f] [-S source-domainname] [-h source-hostfile] [-D destination-domainname] [-H destination-hostfile] [-j num-of-processes] [-J num-of-processes] [-M limit-byte] [-z minimum-byte] [-Z maximum-byte] [-w way-of-scheduling] [-W kilobytes-for-threshold] [-s KB/s-to-simulate] [-F num-for-readahead] [-b bufsize] source-path destination-path

DESCRIPTION

gfpcopy copies files in parallel.

When the source-path parameter is a directory, files under the directory will be copied recursively.

When the destination-path does not exist, the directory is created. When the destination-path exists, a directry of the same name as the source-path is created under the destination-path.

A set of source/destination hosts can be specified by a domain name and/or a hostlist file. When both a domain name and a hostlist file are specified, a set of hosts is determined by both conditions. When a set of source hosts is specified, only files stored on the source hosts are cppied. When a set of source hosts are specified, only files stored on the set of source hosts are copied. A set of hosts can be specified by a domain name and/or a host file. When both a domain name and a host file are specified, a set of hosts is determined by both conditions.

gfpcopy can also read the directory and get information of the entries in parallel.

SOURCE PATH

source-path must be following format. Files on HPSS cannot be copied.

gfarm:...

is Gfarm URL.

file:...

is URL of Local Filesystem.

ANY

is relative path or absolute path of Local Filesystem. When the path is a mountpoint on gfarm2fs, files are copied without passing through the gfarm2fs.

DESTINATION PATH

destination-path must be following format.

gfarm:...

is Gfarm URL of a directory.

file:...

is URL of a directory on Local Filesystem.

hpss:...

is URL of a directory on HPSS. If A directory named the same as the source-path exists under this directory, the source-path cannot be copied. The difference of files cannot be copied to HPSS.

ANY

is relative path or absolute path of Local Filesystem. When the path is a mountpoint on gfarm2fs, files are copied without passing through the gfarm2fs.

GFPCOPY OPTIONS

These are the options only for gfpcopy.

-b bufsize

Specifies the buffer size to copy.

The default value is 64KiB (64 * 1024).

-f

With -f option, existing destination files will be overwritten when the size is different or the modification time (mtime) is different from the source file.

Without -f option, existing destination files will be overwritten when that they are older than the corresponding source files in the modification time.

COMMON OPTIONS

gfpcopy has the same options with gfprep. (see man gfprep)

EXAMPLES

To copy files under the directory recursively.


$ gfpcopy gfarm:///dir file:///tmp/dir


$ gfpcopy file:///tmp/dir gfarm:///dir


$ gfpcopy gfarm:///dir1 gfarm:///dir2


$ gfpcopy gfarm:///dir hpss:///tmp/dir


$ cd /mnt/gfarm2fs
$ gfpcopy dir /tmp/dir

To copy a file.


$ gfpcopy gfarm:///dir/file file:///dir


$ cd /mnt/gfarm2fs
$ gfpcopy file /tmp/dir

NOTES

If the gfmd and gfpcopy are far apart, the gfpcopy operation is slow. Specifying a large value by -j or -J options may be effective in such a case.

SEE ALSO

gfprep(1), gfreg(1), gfexport(1), gfarm2.conf(5)