| Title: | Native and Extensible R Driver for 'Zarr' |
|---|---|
| Description: | The 'Zarr' specification is widely used to build libraries for the storage and retrieval of n-dimensional array data from data stores ranging from local file systems to the cloud. This package is a native 'Zarr' implementation in R with support for all required features of 'Zarr' version 3. It is designed to be extensible such that new stores, codecs and extensions can be added easily. |
| Authors: | Patrick Van Laake [aut, cre, cph] |
| Maintainer: | Patrick Van Laake <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.4.0.9000 |
| Built: | 2026-06-07 16:14:57 UTC |
| Source: | https://github.com/r-cf/zarr |
This method can be used to retrieve a group or array from the Zarr object by its path.
## S3 method for class 'zarr' x[[i]]## S3 method for class 'zarr' x[[i]]
x |
A |
i |
The path to a group or array in |
An instance of zarr_group or zarr_array, or NULL if the path is
not found.
z <- create_zarr() z[["/"]]z <- create_zarr() z[["/"]]
This method can be used to retrieve a group or array from the Zarr group by a relative path to the desired group or array.
## S3 method for class 'zarr_group' x[[i]]## S3 method for class 'zarr_group' x[[i]]
x |
A |
i |
The path to a group or array in |
An instance of zarr_group or zarr_array, or NULL if the path is
not found.
z <- create_zarr() z$add_group("/", "tst") z$add_group("/tst", "subtst") tst <- z[["/tst"]] tst[["subtst"]]z <- create_zarr() z$add_group("/", "tst") z$add_group("/tst", "subtst") tst <- z[["/tst"]] tst[["subtst"]]
This class builds the metadata document for an array to be created or modified. It can also be used to inspect the metadata document of an existing Zarr array.
The Zarr core specification is quite complex for arrays, including codecs and storage transformers that are part optional, part mandatory, and dependent on each other. On top of that, extensions defined outside of the core specification must also be handled in the same metadata document. This class helps construct a valid metadata document, with support for (some) extensions. (If you need support for a specific extension, open an issue on Github.)
When creating array definitions, the default is to use R ordering, meaning that a transpose codec is added with the "order" parameters having the dimensions in reverse order. If you want to use a different ordering, for instance to have a Zarr store with maximum portability, delete the default transpose codec or set its ordering to the desired values. Note that the shape and chunk_shape parameters are set in reference to the ordering in the transpose codec (or the default 0, 1, ... is no transpose codec is present), but that within the R environment the shape and chunk shape are always set to R ordering. This is necessary to be able to apply array operations on the data in R.
This class does not care about the "chunk_key_encoding" parameter. This is addressed at the level of the store.
The "codecs" parameter has a default first codec of "transpose". This ensures that R matrices and arrays can be stored in native column-major order with the store still accessible to environments that use row-major order by default, such as Python. A second default codec is "bytes" that records the endianness of the data. Other codecs may be added by the user, such as a compression codec.
This class only handles the mandatory attributes in a Zarr array metadata document. Optional arguments may be set directly on the Zarr array after it has been created.
formatThe Zarr format to build the metadata for. The value must be 3. After changing the format, many fields will have been reset to a default value.
portableLogical flag to indicate if the array is specified for
maximum portability across environments (e.g. Python, Java, C++).
Default is FALSE. Setting the portability to TRUE implies that R
data will be permuted before writing the array to the store. A value of
FALSE is therefore more efficient.
data_typeThe data type of the Zarr array. After changing the format, many fields will have been reset to a default value.
fill_valueThe value in the array of uninitialized data elements.
The fill_value has to agree with the data_type of the array.
shapeThe shape of the Zarr array, an integer vector of lengths along the dimensions of the array. Setting the shape will reset the chunking settings to their default values.
chunk_shapeThe shape of each individual chunk in which to store
the Zarr array. When setting, pass in an integer vector of lengths of
the same size as the shape of the array. The shape of the array must
be set before setting this. When reading, returns an instance of class
chunk_grid_regular.
codec_info(read-only) Retrieve a data.frame of registered codec
modes and names for this array.
codecs(read-only) A list with validated and instantiated codecs for processing data associated with this array.
array_builder$new()Create a new instance of the array_builder class.
Optionally, a metadata document may be passed in as an argument to
inspect the definition of an existing Zarr array, or to use as a
template for a new metadata document.
array_builder$new(metadata = NULL)
metadataOptional. A JSON metadata document or list of metadata from an existing Zarr array. This document will not be modified through any operation in this class.
An instance of this class.
array_builder$print()Print the array metadata to the console.
array_builder$print()
array_builder$metadata()Retrieve the metadata document to create a Zarr array.
array_builder$metadata(format = "list")
formatEither "list" or "JSON".
The metadata document in the requested format.
array_builder$add_codec()Adds a codec at the end of the currently registered codecs.
Optionally, the .position argument may be used to indicate a specific
position of the codec in the list. Codecs can only be added if their
mode agrees with the mode of existing codecs - if this codec does not
agree with the existing codecs, a warning will be issued and the new
codec will not be registered.
array_builder$add_codec(codec, configuration, .position = NULL)
codecThe name of the codec. This must be a registered codec with an implementation that is available from this package.
configurationList with configuration parameters of the codec.
May be NULL or list() for codecs that do not have configuration
parameters.
.positionOptional, the 1-based position where to insert the codec in the list. If the number is larger than the list, the codec will be appended at the end of the list of codecs.
Self, invisibly.
array_builder$remove_codec()Remove a codec from the list of codecs for the array. A codec cannot be removed if the remaining codecs do not form a valid chain due to mode conflicts.
array_builder$remove_codec(codec)
codecThe name of the codec to remove, a single character string.
array_builder$is_valid()This method indicates if the current specification results in a valid metadata document to create a Zarr array.
array_builder$is_valid()
TRUE if a valid metadata document can be generated, FALSE
otherwise.
These operators can be used to extract or replace data from an array by indices. Normal R array selection rules apply. The only limitation is that the indices have to be consecutive.
## S3 method for class 'zarr_array' x[i, j, ..., drop = TRUE]## S3 method for class 'zarr_array' x[i, j, ..., drop = TRUE]
x |
A |
i, j, ...
|
Indices specifying elements to extract or replace. Indices are
numeric, empty (missing) or |
drop |
If |
When extracting data, a vector, matrix or array, having dimensions as specified in the indices.
x <- array(1:100, c(10, 10)) z <- as_zarr(x) arr <- z[["/"]] arr[3:5, 7:9]x <- array(1:100, c(10, 10)) z <- as_zarr(x) arr <- z[["/"]] arr[3:5, 7:9]
This function creates a Zarr object from an R vector, matrix or array. Default settings will be taken from the R object (data type, shape). Data is chunked into chunks of length 100 (or less if the array is smaller) and compressed.
as_zarr(x, name = NULL, location = NULL)as_zarr(x, name = NULL, location = NULL)
x |
The R object to convert. Must be a vector, matrix or array of a numeric, character or logical type. |
name |
Optional. The name of the Zarr array to be created. If omitted, an array will be created at the root of the Zarr store. |
location |
Optional. If supplied, either an existing zarr_group in a
Zarr object, or a character string giving the location on a local file
system where to persist the data. If the argument is a |
If the location argument is a zarr_group, the new Zarr array is
returned. Otherwise, the zarr object that is newly created and which
contains the Zarr array, or an error if the zarr object could not be
created.
x <- array(1:400, c(5, 20, 4)) z <- as_zarr(x) zx <- array(1:400, c(5, 20, 4)) z <- as_zarr(x) z
This class implements the regular chunk grid for Zarr arrays. It manages reading from and writing to Zarr stores, using the codecs for data transformation.
zarr_extension -> chunking -> chunk_grid_regular
codecsThe list of codecs used by the chunking scheme. These are set by the array when starting to use chunking for file I/O. Upon reading, the list of registered codecs.
chunk_grid_regular$new()Initialize a new chunking scheme for an array.
chunk_grid_regular$new(array_shape, chunk_shape)
array_shapeInteger vector of the array dimensions. This may be
NA for a scalar array.
chunk_shapeInteger vector of the dimensions of each chunk. Ignored for a scalar array.
An instance of chunk_grid_regular.
chunk_grid_regular$print()Print a short description of this chunking scheme to the console.
chunk_grid_regular$print()
Self, invisibly.
chunk_grid_regular$metadata_fragment()Return the metadata fragment that describes this chunking scheme.
chunk_grid_regular$metadata_fragment()
A list with the metadata of this chunking scheme.
chunk_grid_regular$read()Read data from the Zarr array into an R object. The read can
span multiple chunks. Reads will be parallelised if
future::plan(multisession) is set; by default the reading is
sequential.
chunk_grid_regular$read(start, stop)
start, stopInteger vectors of the same length as the dimensionality of the Zarr array, indicating the starting and ending (inclusive) indices of the data along each axis. These are ignored if the Zarr array is a scalar.
A vector, matrix or array of data.
chunk_grid_regular$write()Write data to the array. Writing data always uses a sequential plan.
chunk_grid_regular$write(data, start, stop)
dataAn R object with the same dimensionality as the Zarr array.
start, stopInteger vectors of the same length as the dimensionality of the Zarr array, indicating the starting and ending (inclusive) indices of the data along each axis.
Self, invisibly.
This class implements the sharded chunk grid for Zarr arrays. It manages reading from Zarr stores, using the codecs for data transformation included in the sharding configuration. Writing is not supported with this codec. Storing a scalar array in a sharded grid is not possible either and totally useless anyway.
zarr_extension -> chunking -> chunk_grid_sharded
inner_shape(read-only) The dimensions of each chunk in the shard.
codecs(read-only) The list of codecs used by the sharding scheme.
index_codecs(read-only) The list of codecs used by the sharding scheme for the indexing of the internal chunks.
chunk_grid_sharded$new()Initialize a new sharded chunking scheme for an array.
chunk_grid_sharded$new( array_shape, chunk_shape, inner_shape, index_loc, inner_codecs, index_codecs )
array_shapeInteger vector of the array dimensions.
chunk_shapeInteger vector of the dimensions of each outer chunk, i.e. the size of a shard.
inner_shapeInteger vector of the dimensions of each inner chunk, i.e. the size of a single chunk inside a shard.
index_locLocation of the shard index in the shard file, either "start" or "end".
inner_codecs, index_codecsList of zarr_codec instances to decode
the inner chunks and the index, respectively.
An instance of chunk_grid_sharded.
chunk_grid_sharded$print()Print a short description of this sharded chunking scheme to the console.
chunk_grid_sharded$print()
Self, invisibly.
chunk_grid_sharded$metadata_fragment()Return the metadata fragment that describes this chunking scheme.
chunk_grid_sharded$metadata_fragment()
A list with the metadata of this chunking scheme.
chunk_grid_sharded$read()Read data from the Zarr array into an R object.
chunk_grid_sharded$read(start, stop)
start, stopInteger vectors of the same length as the dimensionality of the Zarr array, indicating the starting and ending (inclusive) indices of the data along each axis.
A vector, matrix or array of data.
This function creates a Zarr v.3 instance, with a store located on the local file system. The root of the Zarr store will be a group to which other groups or arrays can be added.
create_zarr(location)create_zarr(location)
location |
Character string that indicates a location on a file system where the data in the Zarr object will be persisted in a Zarr store in a directory. The character string may contain UTF-8 characters and/or use a file URI format. The Zarr specification recommends that the location use the ".zarr" extension to identify the location as a Zarr store. |
A zarr object.
fn <- tempfile(fileext = ".zarr") my_zarr_object <- create_zarr(fn) my_zarr_object$store$root unlink(fn)fn <- tempfile(fileext = ".zarr") my_zarr_object <- create_zarr(fn) my_zarr_object$store$root unlink(fn)
With this function you can create a skeleton Zarr array from some key properties and a number of derived properties. Compression of the data is set to a default algorithm and level. This function returns an array_builder instance with which you can create directly the Zarr array, or set further properties before creating the array.
define_array(data_type, shape)define_array(data_type, shape)
data_type |
The data type of the Zarr array. |
shape |
An integer vector giving the length along each dimension of the array. |
A array_builder instance with which a Zarr array can be created.
x <- array(1:120, c(3, 8, 5)) def <- define_array("int32", dim(x)) def$chunk_shape <- c(4, 4, 4) z <- create_zarr() # Creates a Zarr object in memory arr <- z$add_array("/", "my_array", def) arr$write(x) arrx <- array(1:120, c(3, 8, 5)) def <- define_array("int32", dim(x)) def$chunk_shape <- c(4, 4, 4) z <- create_zarr() # Creates a Zarr object in memory arr <- z$add_array("/", "my_array", def) arr$write(x) arr
This function opens a Zarr object, connected to a store located on the local file system or on a remote server using the HTTP protocol. The Zarr object can be either v.2 or v.3.
open_zarr(location, read_only = FALSE)open_zarr(location, read_only = FALSE)
location |
Character string that indicates a location on a file system or a HTTP server where the Zarr store is to be found. The character string may contain UTF-8 characters and/or use a file URI format. |
read_only |
Optional. Logical that indicates if the store is to be
opened in read-only mode. Default is |
A zarr object.
fn <- system.file("extdata", "africa.zarr", package = "zarr") africa <- open_zarr(fn) africafn <- system.file("extdata", "africa.zarr", package = "zarr") africa <- open_zarr(fn) africa
Compact display of a regular chunk grid
## S3 method for class 'chunk_grid_regular' str(object, ...)## S3 method for class 'chunk_grid_regular' str(object, ...)
object |
A |
... |
Ignored. |
fn <- system.file("extdata", "africa.zarr", package = "zarr") africa <- open_zarr(fn) tas <- africa[["/tas"]] str(tas$chunking)fn <- system.file("extdata", "africa.zarr", package = "zarr") africa <- open_zarr(fn) tas <- africa[["/tas"]] str(tas$chunking)
Compact display of a sharding chunk grid
## S3 method for class 'chunk_grid_sharded' str(object, ...)## S3 method for class 'chunk_grid_sharded' str(object, ...)
object |
A |
... |
Ignored. |
Compact display of a Zarr object
## S3 method for class 'zarr' str(object, ...)## S3 method for class 'zarr' str(object, ...)
object |
A |
... |
Ignored. |
fn <- system.file("extdata", "africa.zarr", package = "zarr") africa <- open_zarr(fn) str(africa)fn <- system.file("extdata", "africa.zarr", package = "zarr") africa <- open_zarr(fn) str(africa)
Compact display of a Zarr array
## S3 method for class 'zarr_array' str(object, ...)## S3 method for class 'zarr_array' str(object, ...)
object |
A |
... |
Ignored. |
fn <- system.file("extdata", "africa.zarr", package = "zarr") africa <- open_zarr(fn) tas <- africa[["/tas"]] str(tas)fn <- system.file("extdata", "africa.zarr", package = "zarr") africa <- open_zarr(fn) tas <- africa[["/tas"]] str(tas)
Compact display of a Zarr group
## S3 method for class 'zarr_group' str(object, ...)## S3 method for class 'zarr_group' str(object, ...)
object |
A |
... |
Ignored. |
fn <- system.file("extdata", "africa.zarr", package = "zarr") africa <- open_zarr(fn) root <- africa[["/"]] str(root)fn <- system.file("extdata", "africa.zarr", package = "zarr") africa <- open_zarr(fn) root <- africa[["/"]] str(root)
This class implements a Zarr object. A Zarr object is a set of objects that make up an instance of a Zarr data set, irrespective of where it is located. The Zarr object manages the hierarchy as well as the underlying store.
A Zarr object may contain multiple Zarr arrays in a hierarchy. The main
class for managing Zarr arrays is zarr_array. The hierarchy is made up of
zarr_group instances. Each zarr_array is located in a zarr_group.
version(read-only) The version of the Zarr object.
root(read-only) The root node of the Zarr object, usually a zarr_group instance but it could also be a zarr_array instance.
store(read-only) The store of the Zarr object.
domain(read-only) The zarr_domain instance managing the data
in this zarr object.
groups(read-only) Retrieve the paths to the groups of the Zarr object, starting from the root group, as a character vector.
arrays(read-only) Retrieve the paths to the arrays of the Zarr object, starting from the root group, as a character vector.
zarr$new()Create a new Zarr instance. The Zarr instance manages the groups and arrays in the Zarr store that it refers to. This instance provides access to all objects in the Zarr store.
zarr$new(store)
storeAn instance of a zarr_store descendant class where the Zarr objects are located.
A zarr object.
zarr$print()Print a summary of the Zarr object to the console.
zarr$print()
zarr$hierarchy()Print the Zarr hierarchy to the console.
zarr$hierarchy()
zarr$get_node()Retrieve the group or array represented by the node located at the path.
zarr$get_node(path)
pathThe path to the node to retrieve. Must start with a forward-slash "/".
The zarr_group or zarr_array instance located at path, or
NULL if the path was not found.
zarr$add_group()Add a group below a given path.
zarr$add_group(path, name)
pathThe path to the parent group of the new group, a single character string.
nameThe name for the new group, a single character string.
The newly created zarr_group, or NULL if the group could not
be created.
zarr$add_array()Add an array in a group with a given path.
zarr$add_array(path, name, metadata)
pathThe path to the group of the new array, a single character string.
nameThe name for the new array, a single character string.
metadataA list with the metadata for the new array, or a valid
array_builder instance.
The newly created zarr_array, or NULL if the array could not
be created.
zarr$delete_group()Delete a group from the Zarr object. This will also delete
the group from the Zarr store. The root group cannot be deleted but it
can be specified through path = "/" in which case the root group
loses any specific group metadata (with only the basic parameters
remaining), as well as any arrays and sub-groups if recursive = TRUE.
Warning: this operation is irreversible for many stores!
zarr$delete_group(path, recursive = FALSE)
pathThe path to the group.
recursiveLogical, default FALSE. If FALSE, the operation will
fail if the group has any arrays or sub-groups. If TRUE, the group
and all Zarr objects contained by it will be deleted.
Self, invisible.
zarr$delete_array()Delete an array from the Zarr object. If the array is the root of the Zarr object, it will be converted into a regular Zarr object with a root group. Warning: this operation is irreversible for many stores!
zarr$delete_array(path)
pathThe path to the array.
Self, invisible.
zarr$clone()The objects of this class are cloneable with this method.
zarr$clone(deep = FALSE)
deepWhether to make a deep clone.
This class implements a Zarr array. A Zarr array is stored in a node in the hierarchy of a Zarr data set. The array contains the data for an object.
zarr_node -> zarr_array
data_type(read-only) Retrieve the data type of the array.
shape(read-only) Retrieve the shape of the array, an integer vector.
chunking(read-only) The chunking engine for this array.
chunk_separator(read-only) Retrieve the separator to be used for creating store keys for chunks.
codecsThe list of codecs that this array uses for encoding data (and decoding in inverse order).
zarr_array$new()Initialize a new array in a Zarr hierarchy. The array must already exist in the store
zarr_array$new(name, metadata, parent, store)
nameThe name of the array.
metadataList with the metadata of the array.
parentThe parent zarr_group instance of this new array, can be
missing or NULL if the Zarr object should have just this array.
storeThe zarr_store instance to persist data in.
An instance of zarr_array.
zarr_array$print()Print a summary of the array to the console.
zarr_array$print()
zarr_array$hierarchy_nodes()Prints the hierarchy of this array to a character string. Usually called from the Zarr object or a group to display the full group hierarchy.
zarr_array$hierarchy_nodes(idx, total)
idx, totalArguments to control indentation.
zarr_array$read()Read some or all of the array data for the array. For all
types other than logical, any data elements with the fill_value of
the Zarr data type are set to NA.
zarr_array$read(selection)
selectionA list as long as the array has dimensions where each
element is a range of indices along the dimension to write. If missing
or NULL, the entire array will be read.
A vector, matrix or array of data.
zarr_array$write()Write data for the array. The data will be chunked, encoded
and persisted in the store that the array is using. Prior to writing,
any NA values are assigned the fill_value of the data_type of the
Zarr array. Note that the logical type cannot encode NA in Zarr and
any NA values are set to FALSE.
zarr_array$write(data, selection)
dataAn R vector, matrix or array with the data to write. The data in the R object has to agree with the data type of the array.
selectionA list as long as the array has dimensions where each
element is a range of indices along the dimension to write. If missing,
the entire data object will be written.
Self, invisibly.
Zarr codecs encode data from the user data to stored data, using one or more transformations, such as compression. Decoding of stored data is the inverse process, whereby the codecs are applied in reverse order.
zarr_extension -> zarr_codec
mode(read-only) Retrieve the operating mode of the encoding operation of the codec in form of a string "array -> array", "array -> bytes" or "bytes -> bytes".
from(read-only) Character string that indicates the source data type of this codec, either "array" or "bytes".
to(read-only) Character string that indicates the output data type of this codec, either "array" or "bytes".
configuration(read-only) A list with the configuration parameters of the codec, exactly like they are defined in Zarr. This field is read-only but each codec class has fields to set individual parameters.
zarr_codec$new()Create a new codec object.
zarr_codec$new(name, configuration)
nameThe name of the codec, a single character string.
configurationA list with the configuration parameters for this codec.
An instance of this class.
zarr_codec$copy()Create a new, independent copy of this codec.
zarr_codec$copy()
This method always throws an error.
zarr_codec$print()Print a summary of the codec to the console.
zarr_codec$print()
zarr_codec$metadata_fragment()Return the metadata fragment that describes this codec.
zarr_codec$metadata_fragment()
A list with the metadata of this codec.
zarr_codec$encode()This method encodes a data object but since this is the base codec class the "encoding" is a no-op.
zarr_codec$encode(data)
dataThe data to be encoded.
The encoded data object, unaltered.
zarr_codec$decode()This method decodes a data object but since this is the base codec class the "decoding" is a no-op.
zarr_codec$decode(data)
dataThe data to be decoded.
The decoded data object, unaltered.
The Zarr "blosc" codec offers a number of compression options to reduce the size of a raw vector prior to storing, and decompressing when reading.
zarr_extension -> zarr_codec -> zarr_codec_blosc
cnameSet or retrieve the name of the compression algorithm. Must be one of "blosclz", "lz4", "lz4hc", "zstd" or "zlib".
clevelSet or retrieve the compression level. Must be an integer between 0 (no compression) and 9 (maximum compression).
shuffleSet or retrieve the data shuffling of the compression algorithm. Must be one of "shuffle", "noshuffle" or "bitshuffle".
typesizeSet or retrieve the size in bytes of the data type being compressed. It is highly recommended to leave this at the automatically determined value.
blocksizeSet or retrieve the size in bytes of the blocks being compressed. It is highly recommended to leave this at a value of 0 such that the blosc library will automatically determine the optimal value.
zarr_codec_blosc$new()Create a new "blosc" codec object. The typesize argument is
taken from the data type of the array passed in through the data_type
argument and the shuffle argument is chosen based on the data_type.
zarr_codec_blosc$new(data_type, configuration = NULL)
data_typeThe zarr_data_type instance of the Zarr array that this codec is used for.
configurationOptional. A list with the configuration parameters for this codec. If not given, the default compression of "zstd" with level 1 will be used.
An instance of this class.
zarr_codec_blosc$copy()Create a new, independent copy of this codec.
zarr_codec_blosc$copy()
An instance of zarr_codec_blosc.
zarr_codec_blosc$encode()This method compresses a data object using the "blosc" compression library.
zarr_codec_blosc$encode(data)
dataThe raw vector to be compressed.
A raw vector with compressed data.
zarr_codec_blosc$decode()This method decompresses a data object using the "blosc" compression library.
zarr_codec_blosc$decode(data)
dataThe raw vector to be decoded.
A raw vector with the decoded data.
The Zarr "bytes" codec encodes an R data object to a raw byte string, and decodes a raw byte string to a R object, possibly inverting the endianness of the data in the operation.
zarr_extension -> zarr_codec -> zarr_codec_bytes
endianSet or retrieve the endianness of the storage of the data with this codec. A string with value of "big" or "little".
zarr_codec_bytes$new()Create a new "bytes" codec object.
zarr_codec_bytes$new(data_type, chunk_shape, configuration = NULL)
data_typeThe zarr_data_type instance of the Zarr array that this codec is used for.
chunk_shapeThe shape of a chunk of data of the array, an integer vector.
configurationOptional. A list with the configuration parameters
for this codec. The element endian specifies the byte ordering of the
data type of the Zarr array. A string with value "big" or "little". If
not given, the default endianness of the platform is used.
An instance of this class.
zarr_codec_bytes$copy()Create a new, independent copy of this codec.
zarr_codec_bytes$copy()
An instance of zarr_codec_bytes.
zarr_codec_bytes$metadata_fragment()Return the metadata fragment that describes this codec.
zarr_codec_bytes$metadata_fragment()
A list with the metadata of this codec.
zarr_codec_bytes$encode()This method writes an R object to a raw vector in the data type of the Zarr array.
zarr_codec_bytes$encode(data)
dataThe data to be encoded.
A raw vector with the encoded data object.
zarr_codec_bytes$decode()This method takes a raw vector and converts it to an R object of an appropriate type.
zarr_codec_bytes$decode(data)
dataThe data to be decoded.
An R object with the shape of a chunk from the array.
The Zarr "CRC32C" codec computes a 32-bit checksum of a raw vector. Upon encoding the codec appends the checksum to the end of the vector. When decoding, the final 4 bytes from the raw vector are extracted and compared to the checksum of the remainder of the raw vector - if the two don't match a warning is generated.
zarr_extension -> zarr_codec -> zarr_codec_crc32c
zarr_codec_crc32c$new()Create a new "crc32c" codec object.
zarr_codec_crc32c$new()
An instance of this class.
zarr_codec_crc32c$copy()Create a new, independent copy of this codec.
zarr_codec_crc32c$copy()
An instance of zarr_codec_crc32c.
zarr_codec_crc32c$encode()This method computes the CRC32C checksum of a data object and appends it to the data object.
zarr_codec_crc32c$encode(data)
dataA raw vector whose checksum to compute.
The input data raw vector with the 32-bit checksum appended to
it.
zarr_codec_crc32c$decode()This method extracts the CRC32C checksum from the trailing 32-bits of a data object. It then computes the CRC32C checksum from the data object (less the trailing 32-bits) and compares the two values. If the values differ, a warning will be issued.
zarr_codec_crc32c$decode(data)
dataThe raw vector whose checksum to verify.
The data raw vector with the trailing 32-bits removed.
The Zarr "gzip" codec compresses a raw vector prior to storing, and uncompresses the raw vector when reading.
zarr_extension -> zarr_codec -> zarr_codec_gzip
levelThe compression level of the gzip codec, an integer value between 0L (no compression) and 9 (maximum compression).
zarr_codec_gzip$new()Create a new "gzip" codec object.
zarr_codec_gzip$new(configuration = NULL)
configurationOptional. A list with the configuration parameters
for this codec. The element level specifies the compression level of
this codec, ranging from 0 (no compression) to 9 (maximum compression).
An instance of this class.
zarr_codec_gzip$copy()Create a new, independent copy of this codec.
zarr_codec_gzip$copy()
An instance of zarr_codec_gzip.
zarr_codec_gzip$encode()This method encodes a data object.
zarr_codec_gzip$encode(data)
dataThe data to be encoded.
The encoded data object.
zarr_codec_gzip$decode()This method decodes a data object.
zarr_codec_gzip$decode(data)
dataThe data to be decoded.
The decoded data object.
The Zarr sharding codec is not a true codec in the sense that it does not encode or decode - that is left up to regular codec defined inside this "codec" configuration. This implementation can read from a store using sharding, writing is not supported.
zarr_extension -> zarr_codec -> zarr_codec_sharding
zarr_codec_sharding$new()Create a new "crc32c" codec object.
zarr_codec_sharding$new(configuration)
configurationOptional. A list with the configuration parameters for this codec but since this codec doesn't have any the argument is always ignored.
An instance of this class.
zarr_codec_sharding$copy()Create a new, independent copy of this codec.
zarr_codec_sharding$copy()
An instance of zarr_codec_sharding.
The Zarr "transpose" codec registers the storage order of a data object relative to the canonical row-major ordering of Zarr. If the registered ordering is different from the native ordering on the platform where the array is being read, the data object will be permuted upon reading.
R data is arranged in column-major order. The most efficient storage arrangement between Zarr and R is thus column-major ordering, avoiding encoding to the canonical row-major ordering during storage and decoding to column-major ordering during a read. If the storage arrangement is not row-major ordering, a transpose codec must be added to the array definition. Note that within R, both writing and reading are no-ops when data is stored in column-major ordering. On the other hand, when no transpose codec is defined for the array, there will be an automatic transpose of the data on writing and reading to maintain compatibility with the Zarr specification. Using the array_builder will automatically add the transpose codec to the array definition.
For maximum portability (e.g. with Zarr implementations outside of R that do not implement the transpose codec), data should be stored in row-major order, which can be achieved by not including this codec in the array definition.
zarr_extension -> zarr_codec -> zarr_codec_transpose
orderSet or retrieve the 0-based ordering of the dimensions of the array when storing
zarr_codec_transpose$new()Create a new "transpose" codec object.
zarr_codec_transpose$new(shape_length, configuration = list())
shape_lengthThe length of the shape of the array that this codec operates on.
configurationOptional. A list with the configuration parameters
for this codec. The element order specifies the ordering of the
dimensions of the shape relative to the Zarr canonical arrangement. An
integer vector with a length equal to argument
shape_length. The ordering must be 0-based. If not given, the default R
ordering is used.
An instance of this class.
zarr_codec_transpose$copy()Create a new, independent copy of this codec.
zarr_codec_transpose$copy()
An instance of zarr_codec_transpose.
zarr_codec_transpose$encode()This method permutes a data object to match the desired dimension ordering.
zarr_codec_transpose$encode(data)
dataThe data to be permuted, an R matrix or array.
The permuted data object, a matrix or array in Zarr store dimension order.
zarr_codec_transpose$decode()This method permutes a data object from a Zarr store to an R matrix or array.
zarr_codec_transpose$decode(data)
dataThe data to be permuted, from a Zarr store.
The permuted data object, an R matrix or array.
The Numpy UCS-4 format is a fixed-length character string format
where shorter string are padded on the right with 0's. This is not a Zarr
codec but specific to Numpy. It is included here because many Zarr v.2
stores have been written with this formatting of character strings. Since
it does not use a codec in Zarr v.2, it is an invalid configuration in Zarr
v.3 and this package because it embodies the array -> bytes step that is
mandatory in Zarr v.3 - this is why this mock codec is included. This
"codec" encodes an R character object to a raw byte string, and decodes a
raw byte string to a R character object. The character object, typically a
vector but possibly a matrix or array, should use UTF-8 encoding, which is
the standard on modern platforms running R.
This codec is not part of the Zarr v.3 core specification but a commonly
used process in Zarr v.2 on Python to serialize character strings into Zarr
chunks. This implementation enables the use of the Zarr v.2 "<U*" data
type. As a consequence, this codec can only decode data - new data is not
written in this format.
zarr_extension -> zarr_codec -> zarr_codec_ucs4
endian(read-only) Retrieve the endianness of the storage of the data with this codec. A string with value of "big" or "little".
zarr_codec_ucs4$new()Create a new UCS-4 codec object.
zarr_codec_ucs4$new(chunk_shape, configuration)
chunk_shapeThe shape of a chunk of data of the array, an integer vector.
configurationA list with the configuration parameters for this
codec. This is a list created in this package, it does not exist in the
Zarr store as the Numpy UCS-4 method is not a real codec. The element
endian specifies the byte ordering of the data type of the Zarr
array. A string with value "big" or "little". The element width given
the fixed padded string width.
An instance of this class.
zarr_codec_ucs4$copy()Create a new, independent copy of this codec.
zarr_codec_ucs4$copy()
An instance of zarr_codec_ucs4.
zarr_codec_ucs4$metadata_fragment()Return the metadata fragment that describes this codec.
zarr_codec_ucs4$metadata_fragment()
A list with the metadata of this codec, just the name.
zarr_codec_ucs4$decode()This method takes a raw UCS-4 vector and converts it to an R character object.
zarr_codec_ucs4$decode(data)
dataThe data to be decoded.
An R character object with the shape of a chunk from the array.
The Zarr "vlen-utf8" codec encodes an R character object to a raw byte string, and decodes a raw byte string to a R character object. The character object, typically a vector but possibly a matrix or array, should use UTF-8 encoding, which is the standard on modern platforms running R.
This codec is not part of the Zarr v.3 core specification but a commonly
used codec to serialize character strings into Zarr chunks. It is defined
for Zarr v.2. This implementation enables the use of the Zarr v.3
registered "string" data type, as well as Zarr v.2 "|O" data type.
The codec does not handle NA values. On encoding, NA values become
empty strings (""); on decoding empty strings are preserved (not set to
NA). This behaviour is adopted from Python, making it the most
interoperable arrangement. If support for NA values is needed at the
application level, use should be made of a sentinel character string (like
"NO_DATA") which then gets set to NA in the application. This will
obviously not be interoperable, at least not outside of the application
ecosystem.
zarr_extension -> zarr_codec -> zarr_codec_vlenutf8
zarr_codec_vlenutf8$new()Create a new "vlen-utf8" codec object.
zarr_codec_vlenutf8$new()
An instance of this class.
zarr_codec_vlenutf8$copy()Create a new, independent copy of this codec.
zarr_codec_vlenutf8$copy()
An instance of zarr_codec_vlenutf8.
zarr_codec_vlenutf8$metadata_fragment()Return the metadata fragment that describes this codec.
zarr_codec_vlenutf8$metadata_fragment()
A list with the metadata of this codec, just the name.
zarr_codec_vlenutf8$encode()This method writes an R character object to a raw vector.
Prior to writing, any NA values are converted to an empty string.
zarr_codec_vlenutf8$encode(data)
dataThe data to be encoded.
A raw vector with the encoded data object.
zarr_codec_vlenutf8$decode()This method takes a raw vector and converts it to an R character object.
zarr_codec_vlenutf8$decode(data)
dataThe data to be decoded.
An R character object with the shape of a chunk from the array.
This class provides the codec for "zstd" compression.
zarr_extension -> zarr_codec -> zarr_codec_zstd
levelThe compression level of the zstd codec, an integer value between 1L (fast) and 20 (maximum compression).
zarr_codec_zstd$new()Create a new "zstd" codec object.
zarr_codec_zstd$new(configuration = NULL)
configurationOptional. A list with the configuration parameters
for this codec. The element level specifies the compression level of
this codec, ranging from 1 (no compression) to 20 (maximum compression).
An instance of this class.
zarr_codec_zstd$copy()Create a new, independent copy of this codec.
zarr_codec_zstd$copy()
An instance of zarr_codec_zstd.
zarr_codec_zstd$encode()This method encodes a raw data object.
zarr_codec_zstd$encode(data)
dataThe raw data to be encoded.
The encoded raw data object.
zarr_codec_zstd$decode()This method decodes a raw data object.
zarr_codec_zstd$decode(data)
dataThe raw data to be decoded.
The decoded raw data object.
This class implements the "ref" convention. This convention provides a standard way of referring to objects from a referring group or array in a Zarr store. The referenced object may be located in the same Zarr store or in an external Zarr store. In particular, the following convention is implemented here:
{
"schema_url": "https://raw.githubusercontent.com/R-CF/zarr_convention_ref/main/schema.json",
"spec_url": "https://raw.githubusercontent.com/R-CF/zarr_convention_ref/main/README.md",
"uuid": "d89b30cf-ed8c-43d5-9a16-b492f0cd8786",
"name": "ref",
"description": "Referencing Zarr objects external to the current Zarr object"
}
zarr_convention -> zarr_conv_ref
uriThe "uri" field, a character string of an external Zarr store. The URI must follow RFC 3986 and preferably points to a locatable resource like a file on a file system or a store on a web site that is accessible to the same process that opened up the Zarr store having this reference.
nodeThe "node" field, a character string giving the path to a group or array in the current Zarr store or in the store pointed at by the "uri" field.
attributeThe "attribute" field, a character string with a JSON
pointer to a referenced attribute in the metadata of the referenced
node.
zarr_conv_ref$new()Create a new instance of a "ref" convention agent.
zarr_conv_ref$new()
A new instance of a "ref" convention agent.
zarr_conv_ref$write()Write the data of this instance in the attributes of a Zarr object.
zarr_conv_ref$write(attributes)
attributesA list with Zarr attributes for a group or array. The
properties will be written to attributes.
The updated attributes.
zarr_conv_ref$parse_json_pointer()Validate and parse a JSON Pointer (RFC 6901) into its reference tokens.
zarr_conv_ref$parse_json_pointer(ptr)
ptrThe character string from the "attribute" field to parse.
Character vector of raw tokens, or throws an error.
This class implements the "uom" convention. This convention provides a standard way of describing the unit-of-measure of Zarr array data or an attribute. In particular, the following convention is implemented here:
{
"schema_url": "https://raw.githubusercontent.com/clbarnes/zarr-convention-uom/refs/tags/v1/schema.json",
"spec_url": "https://github.com/clbarnes/zarr-convention-uom/blob/v1/README.md",
"uuid": "3bbe438d-df37-49fe-8e2b-739296d46dfb",
"name": "uom",
"description": "Units of measurement for Zarr arrays"
}
zarr_convention -> zarr_conv_uom
versionThe "version" attribute under "ucum", a character string indicating the UCUM vesion that is being used.
unitThe "unit" attribute under "ucum", a character string giving the unit-of-measure in UCUM notation.
descriptionThe "description" attribute, a character string giving a free-text description of the unit-of-measure.
zarr_conv_uom$new()Create a new instance of a "uom" convention agent.
zarr_conv_uom$new()
A new instance of a "uom" convention agent.
zarr_conv_uom$write()Write the data of this instance in the attributes of a Zarr object.
zarr_conv_uom$write(attributes)
attributesA list with Zarr attributes for a group or array. The
properties will be written to attributes.
The updated attributes.
This class implements a basic Zarr convention. A convention is a set of attributes specific to a certain domain of application. These attributes are included in Zarr group and array attributes and are interpreted by application code. Conventions may be grouped in domains and combined with other conventions.
Application-specific conventions need to inherit from this base class and redefine relevant methods. Descendant conventions may have additional methods specific to the domain of the data.
It is not useful to directly instantiate this class, use a descendant convention instead. It is recommended that descendant classes use the "zarr_conv_****" naming pattern and that they are included in a R package with similar conventions and/or domain(s).
name(read-only) The name of the convention, possibly with a trailing semi-colon ":".
schema(read-only) The URL to the schema of the convention.
uuid(read-only) The UUID of the convention, in string format.
specThe URL to the specification of the convention.
descriptionA short description of the convention.
zarr_convention$new()Create a new instance of a Zarr convention agent. This is a "virtual" ancestor class that should not be instantiated directly - instead use one of the descendant classes.
zarr_convention$new(name, schema, uuid)
nameString value with the name of the convention.
schemaString value with the URL to the schema of the convention.
uuidString value with the UUID of the convention.
A new instance of a Zarr convention agent.
zarr_convention$register()Register the use of a convention in the attributes of a Zarr object.
zarr_convention$register(attributes, brief = FALSE)
attributesA list with Zarr attributes for a group or array.
briefLogical flag to indicate if the registration should only include the name and the schema URL or all details (default).
The updated attributes.
zarr_convention$write()Write the data of a convention instance in the attributes of a Zarr object. This method does not do any actual writing. Descendant classes should implement their specific solutions.
zarr_convention$write(attributes)
attributesA list with Zarr attributes for a group or array. The
properties will be written at the root level of attributes.
The updated attributes.
List the Zarr conventions supported by this release
zarr_conventions()zarr_conventions()
A data.frame with descriptions of supported conventions.
This class implements a Zarr data type as an extension point. This class also manages the 'fill_value' attribute associated with the data type.
zarr_extension -> zarr_data_type
data_typeThe data type for the Zarr array, a single character string. Setting the data type will also set the fill value to its default value.
Rtype(read-only) The R data type corresponding to the Zarr data type.
signed(read-only) Flag that indicates if the Zarr data type is signed or not.
size(read-only) The size of the data type, in bytes.
fill_valueThe fill value for the Zarr array, a single value that
agrees with the range of the data_type.
zarr_data_type$new()Create a new data type object.
zarr_data_type$new(data_type, fill_value = NULL)
data_typeThe name of the data type, a single character string.
fill_valueOptionally, the fill value for the data type.
An instance of this class.
zarr_data_type$print()Print a summary of the data type to the console.
zarr_data_type$print()
zarr_data_type$metadata_fragment()Return the metadata fragment for this data type and its fill value.
zarr_data_type$metadata_fragment()
A list with the metadata fragment.
This class implements a basic domain object for Zarr stores. Domains are specific encodings for a certain domain of application. These encodings are included in the attributes of the Zarr groups and arrays and need custom code (usually in the form of another package) to interpret the attributes and properly process array data. Domains can be fully application-specific or they can implement one or more published Zarr conventions.
Domains have to be registered with this package to become available to the processing pipeline. Domain code is then called automatically when a Zarr store is opened for all groups and arrays in the store.
New domains need to inherit from this base class and implement all of the relevant methods. New domains may have additional methods specific to the domain of the data. It is generally not useful to directly instantiate this class: for a Zarr store without a registered domain an instance of this class is initialized automatically.
name(read-only) The name of the domain.
can_read(read-only) Flag to indicate if this domain can read a
Zarr store using this domain. Should always be TRUE.
can_write(read-only) Flag to indicate if this domain can write a
Zarr store using this domain. TRUE for a generic Zarr store; other
domains may report 'FALSE.
zarr_domain$new()Create a new instance of a Zarr domain. This method should
not be called directly (as in zarr_domain$new()); instead,
descendant classes will call this method in their initialization code.
zarr_domain$new(name)
nameCharacter string giving the name of the domain. This may be any sensible string value. The name of the domain must be unique in the session or an error will be thrown upon registering the domain.
A new instance of a domain class.
zarr_domain$build()This method will be called when the domain is requested to asses the Zarr node for domain properties. This method must be implemented by descendant classes and return an appropriate node if it will manage the node. The code should be agile and return swiftly so any non-trivial operations should be left to a later moment, for instance when the node is accessed by the application or end-user.
zarr_domain$build(name, metadata, parent, store)
nameThe name of the node.
metadataList with the metadata of the node.
parentThe parent node of this new node. May be NULL for a root
node.
storeThe store to persist data in.
A zarr_node descendant, typically an instance of a
domain-specific descendant class of zarr_array or zarr_group. If
the domain does not want to manage the node, return FALSE.
List the Zarr domains registered in this session
zarr_domains()zarr_domains()
A list with instances of zarr_domain descendant classes.
Many aspects of a Zarr array are implemented as extensions. More precisely, all core properties of a Zarr array except for its shape are defined as extension points, down to its data type. This class is the basic ancestor for extensions. It supports generation of the appropriate metadata for the extension, as well as processing in more specialized descendant classes.
Extensions can be nested. For instance, a sharding object contains one or more codecs, with both the sharding object and the codec being extension points.
nameThe name of the extension. Setting the name may be restricted by descendant classes.
zarr_extension$new()Create a new extension object.
zarr_extension$new(name)
nameThe name of the extension, a single character string.
An instance of this class.
zarr_extension$metadata_fragment()Return the metadata fragment that describes this extension point object. This includes the metadata of any nested extension objects.
zarr_extension$metadata_fragment()
A list with the metadata of this extension point object.
This class implements a Zarr group. A Zarr group is a node in the hierarchy of a Zarr object. A group is a container for other groups and arrays.
A Zarr group is identified by a JSON file having required metadata,
specifically the attribute "node_type": "group".
zarr_node -> zarr_group
children(read-only) The children of the group. This is a list of
zarr_group and zarr_array instances, or the empty list if the group
has no children.
groups(read-only) Retrieve the paths to the sub-groups of the hierarchy starting from the current group, as a character vector.
arrays(read-only) Retrieve the paths to the arrays of the hierarchy starting from the current group, as a character vector.
zarr_group$new()Open a group in a Zarr hierarchy. The group must already exist in the store.
zarr_group$new(name, metadata, parent, store)
nameThe name of the group. For a root group, this is the empty
string "".
metadataList with the metadata of the group.
parentThe parent zarr_group instance of this new group, can be
missing or NULL for the root group.
storeThe zarr_store instance to persist data in.
An instance of zarr_group.
zarr_group$print()Print a summary of the group to the console.
zarr_group$print()
zarr_group$hierarchy_nodes()Collects the hierarchy of the group and its subgroups and arrays in a character vector. Usually called from the Zarr object or a group to display the full group hierarchy.
zarr_group$hierarchy_nodes(idx = 1L, total = 1L)
idx, totalArguments to control indentation. Should both be 1 (the default) when called interactively. The values will be updated during recursion when there are groups below the current group.
zarr_group$hierarchy()Print the Zarr hierarchy to the console from the current group.
zarr_group$hierarchy()
zarr_group$build_hierarchy()Return the hierarchy contained in the store as a tree of
group and array nodes. This method only has to be called after opening
an existing Zarr store - this is done automatically by user-facing
code. After that, users can access the children property of this
class.
zarr_group$build_hierarchy()
This zarr_group instance with all of its children linked.
zarr_group$get_node()Retrieve the group or array represented by the node located at the path relative from the current group.
zarr_group$get_node(path)
pathThe path to the node to retrieve. The path is relative to the group, it must not start with a slash "/". The path may start with any number of double dots ".." separated by slashes "/" to denote groups higher up in the hierarchy.
The zarr_group or zarr_array instance located at path, or
NULL if the path was not found.
zarr_group$count_arrays()Count the number of arrays in this group, optionally including arrays in sub-groups.
zarr_group$count_arrays(recursive = TRUE)
recursiveLogical flag that indicates if arrays in sub-groups
should be included in the count. Default is TRUE.
zarr_group$add_group()Add a group to the Zarr hierarchy under the current group.
zarr_group$add_group(name)
nameThe name of the new group.
The newly created zarr_group instance, or NULL if the group
could not be created.
zarr_group$add_array()Add an array to the Zarr hierarchy in the current group.
zarr_group$add_array(name, metadata)
nameThe name of the new array.
metadataA list with the metadata for the new array, or an
instance of class array_builder whose data make a valid array
definition.
The newly created zarr_array instance, or NULL if the array
could not be created.
zarr_group$delete()Delete a group or an array contained by this group. When deleting a group it cannot contain other groups or arrays. Warning: this operation is irreversible for many stores!
zarr_group$delete(name)
nameThe name of the group or array to delete. This will also accept a path to a group or array but the group or array must be a node directly under this group.
Self, invisibly.
zarr_group$delete_all()Delete all the groups and arrays contained by this group, including any sub-groups and arrays. Any specific metadata attached to this group is deleted as well - only a basic metadata document is maintained. Warning: this operation is irreversible for many stores!
zarr_group$delete_all()
Self, invisibly.
This class implements a Zarr HTTP store. With this class Zarr stores on web servers can be read. For Zarr v.2 HTTP stores there exists a standard for publishing arrays on the store, using consolidated metadata. This class will look for such metadata in the root of the store. If no consolidated metadata is found then a regular group or array is searched for. Note that if a group is found that there is no standard process to determine what arrays are available in the store and where they are located relative to the root. Typically such information is found in the attributes of the group and you are advised to inspect those attributes and refer to the documentation of the store publisher.
This class performs no sanity checks on any of the arguments passed to the methods, for performance reasons. Since this class should be accessed through group and array objects, it is up to that code to ensure that arguments are valid, in particular keys and prefixes.
zarr_store -> zarr_httpstore
friendlyClassName(read-only) Name of the class for printing.
root(read-only) The root of the HTTP store, identical to its URL.
uri(read-only) The URI of the store location.
separator(read-only) The default chunk separator of the store, usually a slash '/'.
zarr_httpstore$new()Create an instance of this class.
HTTP stores are read-only. Currently two types of Zarr store can be accessed. A Zarr v.2 consolidated metadata file at the root of the store (immediately below the URL) can identify a hierarchy of groups and arrays. Alternatively, a store with a group or a single array, either v.2 or v.3.
zarr_httpstore$new(url)
urlThe path to the HTTP store to be opened. The URL may use UTF-8 code points.
An instance of this class.
zarr_httpstore$exists()Check if a key exists in the store. The key can point to a group, an array, or a metadata file. This check is only relevant for HTTP stores with consolidated metadata. In other cases the single group or array will be at the root.
zarr_httpstore$exists(key)
keyCharacter string. The key that the store will be searched for.
TRUE if argument key is found, FALSE otherwise.
zarr_httpstore$clear()Clearing the store is not supported.
zarr_httpstore$clear()
FALSE.
zarr_httpstore$erase()Removing a key from the store is not supported.
zarr_httpstore$erase(key)
keyIgnored.
FALSE.
zarr_httpstore$erase_prefix()Removing keys from the store is not supported.
zarr_httpstore$erase_prefix(prefix)
prefixIgnored.
FALSE.
zarr_httpstore$list_dir()Retrieve all keys and prefixes with a given prefix and which do not contain the character "/" after the given prefix. In other words, this retrieves all the nodes in the store below the node indicated by the prefix.
zarr_httpstore$list_dir(prefix)
prefixCharacter string. The prefix whose nodes to list.
A character array with all keys found in the store immediately
below the prefix, both for groups and arrays.
zarr_httpstore$list_prefix()Retrieve all keys and prefixes with a given prefix.
zarr_httpstore$list_prefix(prefix)
prefixCharacter string. The prefix whose nodes to list.
A character vector with all paths found in the store below the
prefix location, both for groups and arrays.
zarr_httpstore$set()Storing a (key, value) pair is not supported.
zarr_httpstore$set(key, value)
keyIgnored.
valueIgnored.
Self, invisibly.
zarr_httpstore$set_if_not_exists()Storing a (key, value) pair is not supported.
zarr_httpstore$set_if_not_exists(key, value)
keyIgnored.
valueIgnored.
Self, invisibly.
zarr_httpstore$get()Retrieve the value associated with a given key.
zarr_httpstore$get(key, prototype = NULL, byte_range = NULL)
keyCharacter string. The key for which to get data.
prototypeIgnored. The only buffer type that is supported maps directly to an R raw vector.
byte_rangeIf NULL, all data associated with the key is
retrieved. If a single positive integer, all bytes starting from a
given byte offset to the end of the object are returned. If a single
negative integer, the final bytes are returned. If an integer vector of
length 2, request a specific range of bytes where the end is exclusive.
If the range ends after the end of the object, the entire remainder of
the object will be returned. If the given range is zero-length or
starts after the end of the object, an error will be returned.
A raw vector with the data pointed at by the key.
zarr_httpstore$get_metadata()Retrieve the metadata document of the node at the location
indicated by the prefix argument. The metadata will always be
presented to the caller in the Zarr v.3 format. Attributes, if present,
will be added.
zarr_httpstore$get_metadata(prefix)
prefixThe prefix of the node whose metadata document to retrieve.
A list with the metadata, or NULL if the prefix is not pointing
to a Zarr group or array.
zarr_httpstore$set_metadata()Setting metadata is not supported.
zarr_httpstore$set_metadata(prefix, metadata)
prefixIgnored.
metadataIgnored.
Self, invisible
zarr_httpstore$is_group()Test if path is pointing to a Zarr group.
zarr_httpstore$is_group(path)
pathThe path to test.
TRUE if the path points to a Zarr group, FALSE otherwise.
zarr_httpstore$create_group()Creating a new group in the store is not supported.
zarr_httpstore$create_group(path, name)
path, nameIgnored.
An error indicating that the group could not be created.
zarr_httpstore$create_array()Creating a new array in the store is not supported.
zarr_httpstore$create_array(parent, name, metadata)
parent, name, metadataIgnored.
An error indicating that the array could not be created.
This class implements a Zarr store for the local file system. With this class Zarr stores on devices accessible through the local file system can be read and written to. This includes locally attached drives, removable media, NFS mounts, etc.
The chunking pattern is to locate all the chunks of an array in a single directory. That means that chunks have names like "c0.0.0" in the array directory.
This class performs no sanity checks on any of the arguments passed to the methods, for performance reasons. Since this class should be accessed through group and array objects, it is up to that code to ensure that arguments are valid, in particular keys and prefixes.
zarr_store -> zarr_localstore
friendlyClassName(read-only) Name of the class for printing.
root(read-only) The root directory of the file system store.
uri(read-only) The URI of the store location.
separator(read-only) The default chunk separator of the local file store, usually a dot '.'.
zarr_localstore$new()Create an instance of this class.
If the root location does not exist, it will be created. The location
on the file system must be writable by the process creating the store.
The store is not yet functional in the sense that it is just an empty
directory. Write a root group with .$create_group('/', '') or an
array with .$create_array('/', '', metadata) for a single-array store
before any other operations on the store.
If the root location does exist on the file system it must be a valid Zarr store, as determined by the presence of a "zarr.json" file. It is an error to try to open a Zarr store on an existing location where this metadata file is not present.
zarr_localstore$new(root, read_only = FALSE)
rootThe path to the local store to be created or opened. The path may use UTF-8 code points. Following the Zarr specification, it is recommended that the root path has an extension of ".zarr" to easily identify the location as a Zarr store. When creating a file store, the root directory cannot already exist.
read_onlyFlag to indicate if the store is opened read-only.
Default FALSE.
An instance of this class.
zarr_localstore$exists()Check if a key exists in the store. The key can point to a group, an array, or a chunk.
zarr_localstore$exists(key)
keyCharacter string. The key that the store will be searched for.
TRUE if argument key is found, FALSE otherwise.
zarr_localstore$clear()Clear the store. Remove all keys and values from the store. Invoking this method deletes affected files on the file system and this action can not be undone. The only file that will remain is "zarr.json" or ".zgroup" (version 2) in the root of this store.
zarr_localstore$clear()
TRUE if the operation proceeded, FALSE otherwise.
zarr_localstore$erase()Remove a key from the store. The key must point to an array chunk or an empty group. The location of the key and all of its values are removed.
zarr_localstore$erase(key)
keyCharacter string. The key to remove from the store.
TRUE if the operation proceeded, FALSE otherwise.
zarr_localstore$erase_prefix()Remove all keys in the store that begin with a given prefix. The last location in the prefix is preserved while all keys below are removed from the store. Any metadata extensions added to the group pointed to by the prefix will be deleted as well - only a basic group-identifying metadata file will remain.
zarr_localstore$erase_prefix(prefix)
prefixCharacter string. The prefix to groups or arrays to remove from the store, including in child groups.
TRUE if the operation proceeded, FALSE otherwise.
zarr_localstore$list_dir()Retrieve all keys and prefixes with a given prefix and which do not contain the character "/" after the given prefix. In other words, this retrieves all the nodes in the store below the node indicated by the prefix.
zarr_localstore$list_dir(prefix)
prefixCharacter string. The prefix whose nodes to list.
A character array with all keys found in the store immediately
below the prefix, both for groups and arrays.
zarr_localstore$list_prefix()Retrieve all keys and prefixes with a given prefix.
zarr_localstore$list_prefix(prefix)
prefixCharacter string. The prefix whose nodes to list.
A character vector with all paths found in the store below the
prefix location, both for groups and arrays.
zarr_localstore$set()Store a (key, value) pair. The key points to a specific
file (shard or chunk of an array) in a store, rather than a group or an
array. The key must be relative to the root of the store (so not start
with a "/") and may be composite. It must include the name of the file.
An example would be "group/subgroup/array/c0.0.0". The group hierarchy
and the array must have been created before. If the value exists, it
will be overwritten.
zarr_localstore$set(key, value)
keyThe key whose value to set.
valueThe value to set, a complete chunk of data, a raw vector.
Self, invisibly, or an error.
zarr_localstore$set_if_not_exists()Store a (key, value) pair. The key points to a specific
file (shard or chunk of an array) in a store, rather than a group or an
array. The key must be relative to the root of the store (so not start
with a "/") and may be composite. It must include the name of the file.
An example would be "group/subgroup/array/c0.0.0". The group hierarchy
and the array must have been created before. If the key exists,
nothing will be written.
zarr_localstore$set_if_not_exists(key, value)
keyThe key whose value to set.
valueThe value to set, a complete chunk of data.
Self, invisibly, or an error.
zarr_localstore$get()Retrieve the value associated with a given key.
zarr_localstore$get(key, prototype = NULL, byte_range = NULL)
keyCharacter string. The key for which to get data.
prototypeIgnored. The only buffer type that is supported maps directly to an R raw vector.
byte_rangeIf NULL, all data associated with the key is
retrieved. If a single positive integer, all bytes starting from a
given byte offset to the end of the object are returned. If a single
negative integer, the final bytes are returned. If an integer vector of
length 2, request a specific range of bytes where the end is exclusive.
If the range ends after the end of the object, the entire remainder of
the object will be returned. If the given range is zero-length or
starts after the end of the object, an error will be returned.
An raw vector of data, or NULL if no data was found.
zarr_localstore$get_metadata()Retrieve the metadata document of the node at the location
indicated by the prefix argument. The metadata will always be
presented to the caller in the Zarr v.3 format.
zarr_localstore$get_metadata(prefix)
prefixThe prefix of the node whose metadata document to retrieve.
A list with the metadata, or NULL if the prefix is not pointing
to a Zarr group or array.
zarr_localstore$set_metadata()Set the metadata document of the node at the location
indicated by the prefix argument. The formatting of the metadata
should always use the Zarr v.3 format, it will be converted internally
if the store is Zarr v.2.
zarr_localstore$set_metadata(prefix, metadata)
prefixThe prefix of the node whose metadata document to set.
metadataThe metadata to persist, either a list or an instance
of array_builder.
Self, invisible
zarr_localstore$is_group()Test if path is pointing to a Zarr group.
zarr_localstore$is_group(path)
pathThe path to test.
TRUE if the path points to a Zarr group, FALSE otherwise.
zarr_localstore$create_group()Create a new group in the store under the specified path.
zarr_localstore$create_group(path, name)
pathThe path to the parent group of the new group. Ignored when creating a root group.
nameThe name of the new group. This may be an empty string ""
to create a root group. It is an error to supply an empty string if a
root group or array already exists.
A list with the metadata of the group, or an error if the group could not be created.
zarr_localstore$create_array()Create a new array in the store under the specified path to
the parent argument.
zarr_localstore$create_array(parent, name, metadata)
parentThe path to the parent group of the new array. Ignored when creating a root array.
nameThe name of the new array. This may be an empty string ""
to create a root array. It is an error to supply an empty string if a
root group or array already exists.
metadataA list with the metadata for the array. The list has to
be valid for array construction. Use the array_builder class to
create and or test for validity. An element "chunk_key_encoding" will
be added to the metadata if not already present or with a value other
than a dot "." or a slash "/".
A list with the metadata of the array, or an error if the array could not be created.
https://zarr-specs.readthedocs.io/en/latest/v3/stores/filesystem/index.html
This class implements a Zarr store in RAM memory. With this class Zarr stores can be read and written to. Obviously, any data is not persisted after the memory store is de-referenced and garbage-collected.
All data is stored in a list. The Zarr array itself has a list with the metadata, its chunks have names like "c.0.0.0" and they have an R array-like value.
This class performs no sanity checks on any of the arguments passed to the methods, for performance reasons. Since this class should be accessed through group and array objects, it is up to that code to ensure that arguments are valid, in particular keys and prefixes.
zarr_store -> zarr_memorystore
friendlyClassName(read-only) Name of the class for printing.
separator(read-only) The separator of the memory store, always a dot '.'.
keys(read-only) The defined keys in the store.
zarr_memorystore$new()Create an instance of this class.
zarr_memorystore$new()
An instance of this class.
zarr_memorystore$exists()Check if a key exists in the store. The key can point to a group, an array (having a metadata list as its value) or a chunk.
zarr_memorystore$exists(key)
keyCharacter string. The key that the store will be searched for.
TRUE if argument key is found, FALSE otherwise.
zarr_memorystore$clear()Clear the store. Remove all keys and values from the store. Invoking this method deletes all data and this action can not be undone.
zarr_memorystore$clear()
TRUE. This operation always proceeds successfully once invoked.
zarr_memorystore$erase()Remove a key from the store. The key must point to an array or a chunk. If the key points to an array, the key and all of subordinated keys are removed.
zarr_memorystore$erase(key)
keyCharacter string. The key to remove from the store.
TRUE. This operation always proceeds successfully once invoked,
even if argument key does not point to an existing key.
zarr_memorystore$erase_prefix()Remove all keys in the store that begin with a given prefix.
zarr_memorystore$erase_prefix(prefix)
prefixCharacter string. The prefix to groups or arrays to remove from the store, including in child groups.
TRUE. This operation always proceeds successfully once invoked,
even if argument prefix does not point to any existing keys.
zarr_memorystore$list_dir()Retrieve all keys with a given prefix and which do not contain the character "/" after the given prefix. In other words, this retrieves all the keys in the store below the key indicated by the prefix.
zarr_memorystore$list_dir(prefix)
prefixCharacter string. The prefix whose nodes to list.
A character array with all keys found in the store immediately
below the prefix.
zarr_memorystore$list_prefix()Retrieve all keys and prefixes with a given prefix.
zarr_memorystore$list_prefix(prefix)
prefixCharacter string. The prefix to nodes to list.
A character vector with all paths found in the store below the
prefix location.
zarr_memorystore$set()Store a (key, value) pair. If the value exists, it will
be overwritten.
zarr_memorystore$set(key, value)
keyThe key whose value to set.
valueThe value to set, typically a complete chunk of data, a
raw vector.
Self, invisibly.
zarr_memorystore$set_if_not_exists()Store a (key, value) pair. If the key exists, nothing
will be written.
zarr_memorystore$set_if_not_exists(key, value)
keyThe key whose value to set.
valueThe value to set, a complete chunk of data.
Self, invisibly, or an error.
zarr_memorystore$get()Retrieve the value associated with a given key.
zarr_memorystore$get(key, prototype = NULL, byte_range = NULL)
keyCharacter string. The key for which to get data.
prototypeIgnored. The only buffer type that is supported maps directly to an R raw vector.
byte_rangeIf NULL, all data associated with the key is
retrieved. If a single positive integer, all bytes starting from a
given byte offset to the end of the object are returned. If a single
negative integer, the final bytes are returned. If an integer vector of
length 2, request a specific range of bytes where the end is exclusive.
If the range ends after the end of the object, the entire remainder of
the object will be returned. If the given range is zero-length or
starts after the end of the object, an error will be returned.
An raw vector of data, or NULL if no data was found.
zarr_memorystore$get_metadata()Retrieve the metadata document at the location indicated by
the prefix argument.
zarr_memorystore$get_metadata(prefix)
prefixThe prefix whose metadata document to retrieve.
A list with the metadata, or NULL if the prefix is not pointing
to a Zarr array.
zarr_memorystore$create_group()Create a new group in the store under the specified path.
zarr_memorystore$create_group(path, name)
pathThe path to the parent group of the new group. Ignored when creating a root group.
nameThe name of the new group. This may be an empty string ""
to create a root group. It is an error to supply an empty string if a
root group or array already exists.
A list with the metadata of the group, or an error if the group could not be created.
zarr_memorystore$create_array()Create a new array in the store under key constructed from
the specified path to the parent argument and the name. The key may
not already exist in the store.
zarr_memorystore$create_array(parent, name, metadata)
parentThe path to the parent group of the new array. This is
ignored if the name argument is the empty string.
nameThe name of the new array.
metadataA list with the metadata for the array. The list has to
be valid for array construction. Use the array_builder class to
create and or test for validity. An element "chunk_key_encoding" will
be added to the metadata if it not already there or contains an invalid
separator.
A list with the metadata of the array, or an error if the array could not be created.
This class implements a Zarr node. The node is an element in the hierarchy of the Zarr object. As per the Zarr specification, the node is either a group or an array. Thus, this class is the ancestor of the zarr_group and zarr_array classes. This class manages common features such as names, key, prefixes and paths, as well as the hierarchy between nodes and the zarr_store for persistent storage.
This class should never have to be instantiated or accessed directly.
Instead, use instances of zarr_group or zarr_array. Function arguments
are largely not checked, the group and array instances should do so prior
to calling methods here. The big exception is checking the validity of node
names.
name(read-only) The name of the node.
parent(read-only) The parent of the node. For a root node this
returns NULL, otherwise this zarr_group or zarr_array instance.
store(read-only) The store of the node.
path(read-only) The path of this node, relative to the root node of the hierarchy.
prefix(read-only) The prefix of this node, relative to the root node of the hierarchy.
metadataThe metadata document of this node, a list. CAUTION:
Setting a list that is not properly describing this object will render
the object invalid.
attributes(read-only) Retrieve the list of attributes of this
object. Attributes can be added or modified with the set_attribute()
method or removed with the delete_attributes() method.
zarr_node$new()Initialize a new node in a Zarr hierarchy.
zarr_node$new(name, metadata, parent, store)
nameThe name of the node.
metadataList with the metadata of the node.
parentThe parent node of this new node. Must be omitted when initializing a root node.
storeThe store to persist data in. Ignored if a parent is
specified.
zarr_node$print_attributes()Print the metadata "attributes" to the console. Usually
called by the zarr_group and zarr_array print() methods.
zarr_node$print_attributes(...)
...Arguments passed to embedded functions. Of particular interest
is width = . to specify the maximum width of the columns.
zarr_node$set_attribute()Add an attribute to the metadata of the object. If an
attribute name already exists, it will be overwritten.
zarr_node$set_attribute(name, value)
nameThe name of the attribute. The name must begin with a letter and be composed of letters, digits, and underscores, with a maximum length of 255 characters.
valueThe value of the attribute. This can be of any supported type, including a vector or list of values. In general, an attribute should be a character value, a numeric value, a logical value, or a short vector or list of any of these.
Self, invisibly.
zarr_node$delete_attributes()Delete attributes. If an attribute in name is not present
this method simply returns.
zarr_node$delete_attributes(name)
nameVector of names of the attributes to delete.
Self, invisibly.
zarr_node$save()Persist any edits to the group or array to the store.
zarr_node$save()
Register a Zarr domain for this session
zarr_register_domain(domain)zarr_register_domain(domain)
domain |
An instance of a class descending from |
Nothing.
This class implements a Zarr abstract store. It provides the
basic plumbing for specific implementations of a Zarr store. It implements
the Zarr abstract store interface, with some extensions from the Python
zarr.abc.store.Store abstract class. Functions set_partial_values() and
get_partial_values() are not implemented.
friendlyClassName(read-only) Name of the class for printing.
read_only(read-only) Flag to indicate if the store is read-only.
supports_consolidated_metadataFlag to indicate if the store can consolidate metadata.
supports_deletesFlag to indicate if keys and arrays can be deleted.
supports_listingFlag to indicate if the store can list its keys.
supports_partial_writesDeprecated, always FALSE.
supports_writesFlag to indicate if the store can write data.
version(read-only) The Zarr version of the store.
separator(read-only) The default separator between elements of chunks of arrays in the store. Every store typically has a default which is used when creating arrays. The actual chunk separator being used is determined by looking at the "chunk_key_encoding" attribute of each array.
zarr_store$new()Create an instance of this class. Since this class is "abstract", it should not be instantiated directly - it is intended to be called by descendant classes, exclusively.
zarr_store$new(read_only = FALSE, version = 3L)
read_onlyFlag to indicate if the store is read-only. Default
FALSE.
versionThe version of the Zarr store. By default this is 3.
An instance of this class.
zarr_store$clear()Clear the store. Remove all keys and values from the store.
zarr_store$clear()
Self, invisibly.
zarr_store$erase()Remove a key from the store. This method is part of the abstract store interface in ZEP0001.
zarr_store$erase(key)
keyCharacter string. The key to remove from the store.
Self, invisibly.
zarr_store$erase_prefix()Remove all keys and prefixes in the store that begin with a given prefix. This method is part of the abstract store interface in ZEP0001.
zarr_store$erase_prefix(prefix)
prefixCharacter string. The prefix to groups or arrays to remove from the store, including in child groups.
Self, invisibly.
zarr_store$exists()Check if a key exists in the store.
zarr_store$exists(key)
keyCharacter string. The key that the store will be searched for.
TRUE if argument key is found, FALSE otherwise.
zarr_store$get()Retrieve the value associated with a given key. This method is part of the abstract store interface in ZEP0001.
zarr_store$get(key, prototype, byte_range)
keyCharacter string. The key for which to get data.
prototypeIgnored. The only buffer type that is supported maps directly to an R raw vector.
byte_rangeIf NULL, all data associated with the key is
retrieved. If a single positive integer, all bytes starting from a
given byte offset to the end of the object are returned. If a single
negative integer, the final bytes are returned. If an integer vector of
length 2, request a specific range of bytes where the end is exclusive.
If the range ends after the end of the object, the entire remainder of
the object will be returned. If the given range is zero-length or
starts after the end of the object, an error will be returned.
An raw vector of data, or NULL if no data was found.
zarr_store$getsize()Return the size, in bytes, of a value in a Store.
zarr_store$getsize(key)
keyCharacter string. The key whose length will be returned.
The size, in bytes, of the object.
zarr_store$getsize_prefix()Return the size, in bytes, of all objects found under the group indicated by the prefix.
zarr_store$getsize_prefix(prefix)
prefixCharacter string. The prefix to groups to scan.
The size, in bytes, of all the objects under a group, as a single integer value.
zarr_store$is_empty()Is the group empty?
zarr_store$is_empty(prefix)
prefixCharacter string. The prefix to the group to scan.
TRUE is the group indicated by argument prefix has no
sub-groups or arrays, FALSE otherwise.
zarr_store$list()Retrieve all keys in the store. This method is part of the abstract store interface in ZEP0001.
zarr_store$list()
A character vector with all keys found in the store, both for groups and arrays.
zarr_store$list_dir()Retrieve all keys and prefixes with a given prefix and which do not contain the character "/" after the given prefix. This method is part of the abstract store interface in ZEP0001.
zarr_store$list_dir(prefix)
prefixCharacter string. The prefix to groups to list.
A list with all keys found in the store immediately below the
prefix, both for groups and arrays.
zarr_store$list_prefix()Retrieve all keys and prefixes with a given prefix. This method is part of the abstract store interface in ZEP0001.
zarr_store$list_prefix(prefix)
prefixCharacter string. The prefix to groups to list.
A character vector with all fully-qualified keys found in the store, both for groups and arrays.
zarr_store$set()Store a (key, value) pair.
zarr_store$set(key, value)
keyThe key whose value to set.
valueThe value to set, typically a chunk of data.
Self, invisibly.
zarr_store$set_if_not_exists()Store a key to argument value if the key is not already
present. This method is part of the abstract store interface in
ZEP0001.
zarr_store$set_if_not_exists(key, value)
keyThe key whose value to set.
valueThe value to set, typically an R array.
Self, invisibly.
zarr_store$get_metadata()Retrieve the metadata document of the node at the location
indicated by the prefix argument.
zarr_store$get_metadata(prefix)
prefixThe prefix of the node whose metadata document to retrieve.
zarr_store$set_metadata()Set the metadata document of the node at the location
indicated by the prefix argument. This is a no-op for stores that
have no writing capability. Other stores must override this method.
zarr_store$set_metadata(prefix, metadata)
prefixThe prefix of the node whose metadata document to set.
metadataThe metadata to persist, either a list or an instance
of array_builder.
Self, invisible.
zarr_store$create_group()Create a new group in the store under the specified path to
the parent argument. The parent path must point to a Zarr group.
zarr_store$create_group(parent, name)
parentThe path to the parent group of the new group.
nameThe name of the new group.
A list with the metadata of the group, or an error if the group could not be created.
zarr_store$create_array()Create a new array in the store under the specified path to
the parent argument. The parent path must point to a Zarr group.
zarr_store$create_array(parent, name)
parentThe path to the parent group of the new array.
nameThe name of the new array.
A list with the metadata of the array, or an error if the array could not be created.
https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html#abstract-store-interface
Unregister a Zarr domain from this session
zarr_unregister_domain(domain)zarr_unregister_domain(domain)
domain |
The name of the |
Nothing.