syq map

Print local source/destination pairs as NDJSON for syq cp --mapping:

syq map --srcs-in photos > photos.ndjson
syq cp --mapping photos.ndjson -C photos --into archive

Named selectors must be relative to the source base; use -C DIR or --root DIR to choose that base. --srcs-in must be the only selector when used. --as requires one named object and cannot combine with --srcs-in.

See Rename and reorganize during a copy for worked examples.

syq map [OPTIONS] PATH...
syq map [OPTIONS] --srcs-in DIR

Sources and selection

Destination placement

Help and version

Mapping format

cp --mapping reads one JSON object per line. map emits this format from local sources; other programs can generate it too.

Paths use encoding: "utf-8", or "base64" with standard base64 of raw filename bytes. Absolute or empty paths, and any . or .. component, are refused. Unknown fields are refused too.

expected_hash checks a regular file’s complete contents, including reused bytes. A mismatch fails the entry. See expected hashes for algorithms and behavior with in-place writes.

To set destination attributes without changing the source, add a metadata object, for example "metadata": {"mode": 416, "mtime": 1700000000}. This sets permissions to 0640 and the modification time to the given Unix second. Optional fields are mode (permission bits, 0–4095), numeric uid and gid, mtime (Unix seconds), and mtime_nsec (0–999999999, requires mtime). A supplied mtime defaults to zero fractional seconds. Omitted attributes follow normal copy behavior; the top-level mtime remains informational.

Explicit attributes apply without --preserve. Ownership requests fail if the filesystem refuses them; symlinks cannot have a requested mode. S3 uploads store the attributes in syq object metadata; downloads apply them to the filesystem. S3-to-S3 copies keep other object metadata and stay server-side. Restricted receivers also require matching --preserve permissions in the signed grant. Selection rules such as --only-new still take precedence. Supplied timestamps disable the size/time shortcut for those entries. Use --hash to compare contents when repeating a copy with a fixed destination timestamp. Older binaries that do not support metadata reject the manifest.

Each entry copies one object. A directory entry is not recursive. syq map emits its descendants as separate entries. A missing source or wrong kind fails that entry while independent entries continue.

Mapping restrictions

Use --mapping instead of source selectors, with --into, --into-new, or --into-existing. It cannot combine with --as, --prune, or --detach. Use the same source base for map and cp. --root confines source selection. Follow options apply to supplied paths, never to links traversed by a manifest entry. map refuses non-UTF-8 names; hand-written manifests may use base64.

The whole manifest is validated before copying. Malformed input, duplicate destination names, and declared file/ancestor conflicts refuse the run. The destination container may already have been created. Memory use grows with manifest size. Conflicts discovered while copying fail the affected entries.

For restricted copies between servers, mapped destinations and their parent directories count against the receiver’s entry limit. Each line can be up to 1 MiB, and each destination path up to 4096 bytes. There is no separate total manifest-size limit. Missing parent directories are created; a file or symlink blocking a parent must be moved or removed before retrying.

kind: "special" checks the source type; add --preserve=specials to copy those entries. Use --mapping - for a pipeline; a named FIFO manifest requires Linux with procfs.

Filter a mapping

These jq filters keep or remove entries before cp --mapping reads them:

# Keep files of at least 1 MiB, plus directory and link entries:
jq -c 'select(.kind != "file" or .size >= 1048576)'
# Drop device, FIFO, and socket entries:
jq -c 'select(.kind != "special")'