Comparing AWS and GCP Storage Buckets: A CLI Command Guide

Overview

Compared to AWS management console, I find GCP's management console to be quite overwhelming. By using the command-line interface gcloud, it helped me to focus and map familiar features found in S3 to Google Storage's Cloud Bucket.

AWS's command-line interface aws for S3 have 2 sets of command

1. aws s3api consists of all the low-level operations for the full suite of available actions

2. aws s3 is a wrapper for all the common functions to manage the buckets

Prerequisites

You will need to configure the aws and gcloud command with the required authentication before running the commands listed below

# AWS - Guided to provided the 
# 1. AWS_ACCESS_KEY_ID
# 2. AWS_SECRET_ACCESS_KEY
aws configure

# GCP - Guided to use SSO login 
gcloud auth application-default login

CRUD Operations

Below are a list of actions to manage the buckets in the respective cloud providers, AWS and GCP respectively.

Create new bucket

To start off, replace the value of BUCKET_NAME value to the desired bucket name

# Common variables
BUCKET_NAME="REPLACE_WITH_BUCKET_NAME"

# AWS
aws s3api create-bucket --bucket "$BUCKET_NAME" --region ap-southeast-1 --create-bucket-configuration LocationConstraint=ap-southeast-1

# GCP
gcloud storage buckets create "gs://$BUCKET_NAME" --location="asia"

List all buckets

# AWS
aws s3 ls

# GCP
## By providing the --format option, will only list the bucket names, else other metadata will be provided
gcloud storage buckets list [--format="json(name)"]

List bucket's objects

# AWS
aws s3 ls s3://BUCKET_NAME/

# GCP
gcloud storage ls gs://BUCKET_NAME/

Upload new object

# AWS 
aws s3 cp LOCAL_FILENAME s3://BUCKET_NAME/REMOTE_FILENAME

# GCP
gcloud storage cp LOCAL_FILENAME gs://BUCKET_NAME/REMOTE_FILENAME

Delete object from bucket

# AWS
aws s3 rm s3://BUCKET_NAME/REMOTE_FILENAME

# GCP
gcloud storage rm gs://BUCKET_NAME/REMOTE_FILENAME