Google Cloud Storage as S3 - Case Joplin

I use Joplin as my note-taking app over multiple devices, which makes a shared backend for note-syncing a thing I can't live without. Joplin supports a ton of different backends, including AWS S3. Obviously, as a Google enthusiast, I really really wanted to use Google Cloud Storage as my S3 backend. This was a surprisingly easy thing to do.

Recently Google Cloud has been my first choice for personal cloud projects. It feels like a very developer-centric platform with developer-centric tooling. Arguably, Google seems to think that everyone working in technology is a developer. This is a sentiment I can get behind, as I aspire to be more of a hacker than an enterprise user.

For this curious case of using Cloud Storage in place of S3, Google has built an XML interoperability layer. Getting Joplin to talk to this API was as simple as providing the Cloud Storage URI https://storage.googleapis.com in place of the S3 URL.

TL;DR: You can find the complete Terraform configuration for this setup in this Gist. Following is a breakdown of the key resources and why you need them.

Bucket

First, we need a Cloud Storage bucket for the synchronized Joplin state. I like to use a random suffix to ensure the bucket name is unique, as in the following Terraform snippet.

resource "random_id" "bucket-suffix" {
  byte_length = 8
  keepers = {
    project = var.project
  }
}

resource "google_storage_bucket" "sync" {
  name                     = "joplin-${random_id.bucket-suffix.hex}"
  public_access_prevention = "enforced"
  location                 = var.region
  project                  = var.project
}

Note that I am using Terraform variables project and region throughout these examples.

Service Account

Then, I recommend creating a separate service account with minimum permissions to access the bucket. Good infosec hygiene keeps the setup less smelly.

resource "google_service_account" "sync" {
  account_id   = "sync-bucket"
  display_name = "Joplin sync bucket service account"
}

resource "google_storage_bucket_iam_member" "sync" {
  bucket = google_storage_bucket.sync.name
  role   = "roles/storage.objectUser"
  member = "serviceAccount:${google_service_account.sync.email}"
}

roles/storage.objectUser gives the account permission to do what it likes with objects and folders in the bucket, without allowing it to modify access and permissions. Joplin likes to create and delete both objects and folders, so this is the best fit from predefined roles for GCS.

Access key and secret

Access to S3 is authenticated with an access key and secret. We can generate HMAC keys for the service account and push them to Secrets Manager.

resource "google_storage_hmac_key" "sync" {
  service_account_email = google_service_account.sync.email
}

resource "google_secret_manager_secret" "hmac-secret" {
  secret_id = "hmac-secret"
  replication {
    auto {}
  }
}

resource "google_secret_manager_secret" "hmac-id" {
  secret_id = "hmac-id"
  replication {
    auto {}
  }
}

resource "google_secret_manager_secret_version" "hmac-secret" {
  secret      = google_secret_manager_secret.hmac-secret.id
  secret_data = google_storage_hmac_key.sync.secret
}

resource "google_secret_manager_secret_version" "hmac-id" {
  secret      = google_secret_manager_secret.hmac-id.id
  secret_data = google_storage_hmac_key.sync.access_id
}

Joplin configuration

Finally, we can configure Joplin to use the bucket and HMAC key. The Google parameters match S3 settings pretty closely.

• S3 bucket: output of google_storage_bucket.sync.name.
• S3 URL: storage.googleapis.com (Google's XML API).
• S3 region: your var.region (Google Cloud region).
• The S3 access key: HMAC ID (fetch from Secrets Manager, e.g., gcloud secrets versions access latest --secret hmac-id).
• The S3 access secret: HMAC secret (fetch from Secrets Manager).