Google Cloud Storage

Google Cloud Storage#

Utility functions related to storing neuroglancer states in Google Cloud Storage.

ngsidekick.gcs.upload_ngstates(bucket_dir, states, threads=0, processes=0, disable_cache=False, return_prefix='https://neuroglancer-demo.appspot.com')[source]#

Use multithreading or multiprocessing to upload many files in parallel, similar to gsutil -m cp []…], except that in this case you must choose between multithreading or multiprocessing (not a combination of the two).

For the return values, neuroglancer links are returned (using the given prefix). If no prefix is given, then the URLs to the uploaded files are returned.

Parameters:

  • bucket_dir – str The bucket directory to upload the states to, such as ‘gs://mybucket/mydir’

  • states – dict A dictionary of neuroglancer states, keyed by the filename.

  • threads – int The number of threads to use for uploading.

  • processes – int The number of processes to use for uploading.

  • disable_cache – bool If True, set public, no-store metadata on the uploaded files to disable caching.

  • return_prefix – str If provided, return neuroglancer links using the given neuroglancer deployment. If None, then return the direct URLs to the uploaded files in google storage.

Returns:

list of str

A list of links.

ngsidekick.gcs.upload_ngstate(bucket_path, state, disable_cache=False, return_prefix='https://neuroglancer-demo.appspot.com')[source]#

Upload the given JSON state to a gbucket location, such as ‘gs://flyem-user-links/short/foobar.json’

For the return value, a neuroglancer link is returned (using the given prefix). If no prerix is given, then the URL to the uploaded file is returned.

Parameters:

  • bucket_path – str The bucket directory to upload the state to, such as ‘gs://mybucket/mydir’

  • state – dict The neuroglancer state to upload.

  • disable_cache – bool If True, set public, no-store metadata on the uploaded file to disable caching.

  • return_prefix – str If provided, return neuroglancer links using the given neuroglancer deployment. If None, then return the direct URLs to the uploaded files in google storage.

Returns:

str

A neuroglancer link to the uploaded file or the direct URL to the uploaded file in google storage, depending on whether return_prefix was provided.

ngsidekick.gcs.upload_json(obj, bucket_path, disable_cache=True)[source]#

Upload the given JSON data to a gbucket location, such as ‘gs://flyem-user-links/short/foobar.json’

ngsidekick.gcs.upload_to_bucket(bucket, blob_name, blob_contents, content_type='application/json', disable_cache=False)[source]#

Upload a blob of data to the specified google storage bucket.

Parameters:

  • bucket – str or google.cloud.storage.Bucket The bucket to upload the data to.

  • blob_name – str The name of the blob to upload.

  • blob_contents – str The data to upload.

ngsidekick.gcs.make_bucket_public(bucket=None)[source]#

To host data in google storage for neuroglancer for general access, you need to make the bucket public and configure permissive CORS settings.

This function does that.

This feature is also installed along with ngsidekick as a command-line tool:

make-bucket-public gs://mybucket/mydir
Parameters:

bucket – str or None The bucket to make public, such as ‘gs://mybucket/mydir’ If None, then the bucket is read from the command line.

Contents