Python operations
Module functions syq.cp, syq.rm, syq.map, syq.open_reader, and
syq.open_writer use a default Client.
AsyncClient has the same arguments and result types; await its operations
except map, open_reader, and open_writer, which return async context managers.
For generated data and entry transformations, see Streams and mappings. For returned objects, callbacks, and errors, see Results and events.
Client and executable selection
Client(*, executable=None, cache_dir=None, process_cwd=None, env=None, timeout=None)
and AsyncClient(...) accept:
| Argument | Meaning |
|---|---|
executable | Custom executable path, or name to find on PATH; default: bundled syq |
cache_dir | Opt into a separately downloaded executable at this cache root |
process_cwd | Local subprocess working directory; default: inherit |
env | Subprocess environment mapping; default: inherit |
timeout | Operation timeout in seconds; default: no limit |
client.version() returns the executable version as text.
syq.version(executable=None) does the same without a client.
syq.managed_executable(cache_dir=None) returns the verified executable path,
downloading it if needed. See Compatibility for version selection and caching.
Arguments and validation
Options use CLI names with hyphens replaced by underscores. Python keywords
get a trailing underscore: from_, as_. Paths accept str, bytes, or
os.PathLike; selector keywords accept one path or an iterable of paths.
Use src=["a", "b"] for CLI --srcs a b, and likewise src_non_dir and src_dir.
Shared by cp, rm, and map | Meaning |
|---|---|
*sources, src | Select named objects |
srcs_in | Select a directory’s contents |
src_non_dir, src_dir | Require non-directory objects or directories |
cwd | Source resolution base; may be remote for cp and rm |
root | Confine source resolution beneath this directory; requires relative selectors; conflicts with cwd |
follow, follow_src | Follow source symlinks; follow also enables destination following for cp |
timeout | Omitted: use client default; None: no timeout; number: timeout in seconds |
Boolean flags default to False; other optional arguments default to None,
except check=True and timeout, whose omitted value inherits the client default.
Invalid argument combinations raise SyqInvocationError;
filesystem and remote checks happen in syq.
cp
cp(*sources, **options) → CpResult copies files. Choose one placement option unless every destination is a callback.
In addition to the shared arguments above, it accepts:
| Options | Values / purpose |
|---|---|
from_, to | SSH endpoint strings or s3://BUCKET; omitted endpoints are local |
into, into_new, into_existing | Destination directory paths |
as_, as_new, as_existing | Exact destination paths |
mapping | Mapping, MapStream, manifest path, or iterable of MappingEntry; replaces selectors; conflicts with as_* and prune. Async clients also accept AsyncMapping and async iterables |
stream_concurrency | Maximum callback entries active at once; default 4, range 1..256; transport worker and request limits are shared across entries |
follow_dst | Boolean: follow destination symlinks |
prune, dry_run, hash | Boolean: mirror, preview, or compare content |
integrity_checking | Comma-separated string, e.g. "transfer=sha256"; defaults to size/mtime comparison and no extra payload checks |
only_new | Boolean: copy missing entries without replacing existing ones |
ignore | Pattern string, IgnoreFrom(path), or ordered iterable of either |
ignore_from | Rule file path or iterable of paths; applied after ignore |
preserve | Preservation string or iterable: times, permissions, ownership, specials |
inplace, no_compress | Boolean: update destination files in place or disable compression |
max_delete | Nonnegative integer deletion limit; requires prune=True |
resource_limits | Comma-separated ceilings that keep automatic tuning, e.g. "bandwidth=10M,workers=4"; a concurrency key conflicts with the same key in performance_tuning |
performance_tuning | Comma-separated overrides, e.g. "workers=4" or "s3-objects=4,s3-parts-per-object=8,s3-requests=16"; omitted means automatic |
s3_endpoint, s3_region, s3_profile | Endpoint URL, signing region, and AWS profile strings |
s3_header | Iterable of "NAME: VALUE" strings; applied before signing every request |
auth_from | Credential source string |
coordinate_at, rsh, peer_auth | Coordinator, SSH command, and peer authentication strings |
pscope | Existing ephemeral scope path for forward SSH connection reuse |
syq_path | Remote executable path |
no_bootstrap, tcp_plain, no_tcp | Boolean remote/transport controls |
tcp_ports, tcp_congestion | Port range and congestion-control strings |
receiver_max_entries, receiver_max_bytes | Receiver ceilings: integer entries, native size string or integer bytes |
receiver_receipt | "sizes" or "hashes" |
on_event, results, check | See events and failures |
Option behavior is covered in Copy files, Remote copy details, and Object storage.
For example, client.cp("data", to="s3://bucket", into="backup", s3_header=["X-Tigris-Consistent: true"]) uploads local data using credentials
from the subprocess environment or AWS configuration. Copies between two S3
endpoints use server-side copying within the same service; content verification
options that require reading object bodies are rejected. SSH/S3 combinations
are not supported. S3 results use EndpointKind.S3.
pscope selects an isolated scope for reusing SSH connections. For return
copies or commands, use syq persist connect server and omit pscope. See
persistence in scripts
for setup and cleanup, and
Compatibility
for executable selection.
Typed SSH-to-SSH copies require an enrolled receiver or
coordinate_at="local". With dry_run=True, they require
coordinate_at="local". Use run for detached commands and human output options.
IgnoreFrom(path) is a frozen dataclass holding a rule-file path (str,
bytes, or os.PathLike). To interleave rule files and inline patterns:
ignore=[syq.IgnoreFrom("rules"), "!keep.tmp"]. The last matching rule wins.
rm
Positional sources and src refuse directories. src_dir removes a
tree recursively; srcs_in removes its contents recursively and keeps the root.
on="server" selects the removal endpoint. A final selected symlink is
always unlinked. Both follow_src=True and follow=True permit symlinks in
cwd, root, and selector parent directories.
Directory and contents selectors reject a final symlink even with following
enabled.
rm(*sources, **options) → RmResult removes selected entries. Besides the shared
arguments, it accepts on, dry_run, performance_tuning, syq_path,
no_bootstrap, pscope, on_event, results, and check with the types above.
It supports local, ordinary SSH, and S3 endpoints. Command-restricted receivers
reject removal. See Remove files.
S3 removal uses rm(..., on="s3://bucket") with optional s3_endpoint,
s3_region, s3_profile, s3_header, and auth_from="@NAME". s3_all_versions=True permanently
removes all selected versions and delete markers; s3_version_id="ID" selects
one version of one exact key. These options are mutually exclusive.
RemovalTrace and RemovalResult expose optional s3_version_id and
s3_delete_marker fields. The same arguments work with AsyncClient.rm.
run
client.run(args, *, check=True, cwd=None, env=None, timeout=CLIENT_DEFAULT, input=None)
returns Result. input accepts bytes. args is a sequence of arguments
after the executable name, passed without a shell.
Here, cwd is the local subprocess directory. cwd and env use client
defaults when omitted or None. timeout uses the client default only when
omitted; explicit None disables it. Module-level syq.run takes the same
arguments plus executable=None and defaults to no timeout.
Wrappers can forward syq.CLIENT_DEFAULT to preserve client inheritance.
syq.Timeout is the type alias for a number, None, or that sentinel:
def copy_data(client: syq.Client, *, timeout: syq.Timeout = syq.CLIENT_DEFAULT):
return client.cp("data", into="backup", timeout=timeout)
CLIENT_DEFAULT applies to client methods (including module-level cp, rm,
and map); client constructors and module-level run have no client default
to inherit.
Timeouts cover subprocess execution. Managed installation and mapping-input materialization happen before the copy process starts and are not covered by its timeout. Async cancellation still stops mapping-input preparation.
For syq exec, pass --cwd in the argument list to select the receiving
working directory. The SDK’s cwd= parameter selects the local working
directory of the requesting process. See the
command example.
Result
Frozen dataclass returned by run() and held in SyqProcessError.result:
| Attribute | Type | Meaning |
|---|---|---|
argv | tuple[str | bytes, ...] | Executed argument list, including the executable |
returncode | int | Process exit status |
stdout | bytes | Complete captured standard output |
stderr | bytes | Complete captured standard error |
Compatibility
Python 3.13.4+ on Linux and macOS; no runtime Python dependencies. Each Python
package uses the matching syq release. syq.__version__ and
syq.PINNED_SYQ_VERSION report those versions. Pin the package in your dependency
file to keep the pairing.
Bundled executable
By default, the SDK runs the executable installed with its Python wheel. It
locates that executable through the package’s installation record, without
searching PATH, downloading files, or creating an executable cache. Each
Python environment has its own installation. Removing the package also removes
its executable.
Managed executable
For callers using a separate cache, Client(cache_dir=...) downloads the matching executable on first use and verifies
it against the package’s embedded release manifest. It checks the cached binary
before every use and replaces missing or corrupt entries. It does not search
PATH. syq.managed_executable() uses the same download and verification checks.
The default cache is $XDG_CACHE_HOME/syq/sdk/python/v<version>/ when
XDG_CACHE_HOME is absolute, or ~/.cache/syq/sdk/python/v<version>/ otherwise.
Use Client(cache_dir=...) to change the cache root, or
syq.managed_executable() to download ahead of time and get the path.
Custom executable
Client(executable="/opt/bin/syq") uses that binary;
Client(executable="syq") searches PATH. Overrides bypass bundled selection and managed download
and verification, so you are responsible for compatibility and origin. Typed
calls still validate automation output. A failed executable selection does not
fall back to another binary.