cockroach userfile upload

On this page Carat arrow pointing down
Warning:
CockroachDB v20.2 is no longer supported as of May 10, 2022. For more details, refer to the Release Support Policy.

New in v20.2: The cockroach userfile upload command uploads a file to the user-scoped file storage using a SQL connection.

This command takes in a source file to upload and a destination filename. It will then use a SQL connection to upload the file to the destination.

Note:

A userfile uses storage space in the cluster, and is replicated with the rest of the cluster's data. We recommended using cockroach userfile upload for quick imports from your client (about 15MB or smaller).

Required privileges

The user must have the CREATE privilege on the target database. CockroachDB will proactively grant the user GRANT, SELECT, INSERT, DROP, DELETE on the metadata and file tables.

A user can only upload files to their own user-scoped storage, which is accessed through the userfile URI. CockroachDB will revoke all access from every other user in the cluster except users in the admin role.

Synopsis

Upload a file:

$ cockroach userfile upload <location/of/file> <destination/of/file> [flags]
Note:

You must specify a source path.

View help:

$ cockroach userfile upload --help

File destination

Userfile operations are backed by two tables: files (which holds file metadata) and payload (which holds the file payloads). To reference these tables, you can:

  • Use the default URI: userfile://defaultdb.public.userfiles_$user/.
  • Provide a fully qualified userfile URI that specifies the database, schema, and table name prefix you want to use.

    • If you do not specify a destination URI/path, then CockroachDB will use the default URI scheme and host, and the basename from the source argument as the path. For example: userfile://defaultdb.public.userfiles_root/local
    • If the destination is a well-formed userfile URI (i.e., userfile://db.schema.tablename_prefix/path/to/file), then CockroachDB will use that as the final URI. For example: userfile://foo.bar.baz_root/destination/path
    • If destination is not a well-formed userfile URI, then CockroachDB will use the default userfile URI schema and host (userfile://defaultdb.public.userfiles_$user/), and the destination as the path. For example: userfile://defaultdb.public.userfiles_root/destination/path
Warning:

Userfile is not a filesystem and does not support filesystem semantics. The destination file path must be the same after normalization (i.e., if you pass any path that results in a different path after normalization, it will be rejected).

Note:

Files are uploaded with a .tmp suffix and are renamed once the userfile upload transaction has committed (i.e, the process ends gracefully). Therefore, if a file you believed had finished uploading has a .tmp suffix, then the upload should be retried.

Flags

Flag Description
--cert-principal-map A comma-separated list of <cert-principal>:<db-principal> mappings. This allows mapping the principal in a cert to a DB principal such as node or root or any SQL user. This is intended for use in situations where the certificate management system places restrictions on the Subject.CommonName or SubjectAlternateName fields in the certificate (e.g., disallowing a CommonName like node or root). If multiple mappings are provided for the same <cert-principal>, the last one specified in the list takes precedence. A principal not specified in the map is passed through as-is via the identity function. A cert is allowed to authenticate a DB principal if the DB principal name is contained in the mapped CommonName or DNS-type SubjectAlternateName fields.
--certs-dir The path to the certificate directory containing the CA and client certificates and client key.

Env Variable: COCKROACH_CERTS_DIR
Default: ${HOME}/.cockroach-certs/
--echo-sql Reveal the SQL statements sent implicitly by the command-line utility.
--url A connection URL to use instead of the other arguments.

Env Variable: COCKROACH_URL
Default: no URL
--user
-u
The SQL user that will own the client session.

Env Variable: COCKROACH_USER
Default: root

Examples

Upload a file

To upload a file to the default storage (userfile://defaultdb.public.userfiles_$user/):

icon/buttons/copy
$ cockroach userfile upload /Users/maxroach/Desktop/test-data.csv /test-data.csv --certs-dir=certs
successfully uploaded to userfile://defaultdb.public.userfiles_root/test-data.csv

Also, a file can be uploaded to the default storage if the destination is not specified:

icon/buttons/copy
$ cockroach userfile upload /Users/maxroach/Desktop/test-data2.csv --certs-dir=certs
successfully uploaded to userfile://defaultdb.public.userfiles_root/test-data2.csv

Then, you can use the file to IMPORT or IMPORT INTO data.

Upload a file to a specific directory

To upload a file to a specific destination, include the destination in the command:

icon/buttons/copy
$ cockroach userfile upload /Users/maxroach/Desktop/test-data.csv /test-upload/test-data.csv --cert-dir=certs
successfully uploaded to userfile://defaultdb.public.userfiles_root/test-upload/test-data.csv

Then, you can use the file to IMPORT or IMPORT INTO data.

Upload a file to a non-default userfile URI

icon/buttons/copy
cockroach userfile upload /Users/maxroach/Desktop/test-data.csv userfile://testdb.public.uploads/test-data.csv
successfully uploaded to userfile://testdb.public.uploads/test-data.csv

Known limitation

  • cockroach userfile upload does not not currently allow for recursive uploads from a directory. This feature will be present with the --recursive flag in future versions. Tracking GitHub Issue

See also


Yes No
On this page

Yes No