GCP Cloud Storage Event

This guide will help you set up GCP Cloud Storage event notifications using Terraform Infrastructure as Code.

Prerequisites

GCP Project with billing enabled
gcloud CLI installed
Terminal/Command Line access
Owner or Editor role in the GCP project

Step 1: Install Required Tools

1.1 Install gcloud CLI

Check if already installed:

gcloud --version

For macOS:

curl https://sdk.cloud.google.com | bash
exec -l $SHELL

For Linux:

curl https://sdk.cloud.google.com | bash
exec -l $SHELL

For Windows: Download from: https://cloud.google.com/sdk/docs/install

1.2 Install Terraform

For macOS:

brew tap hashicorp/tap
brew install hashicorp/tap/terraform

For Linux:

wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update && sudo apt install terraform

For Windows: Download from: https://www.terraform.io/downloads

Verify installation:

terraform --version

Step 2: Authenticate with GCP

2.1 Initialize gcloud

gcloud init

This will guide you through:

Logging in to your Google account
Selecting your GCP project
Setting default compute region/zone

gcloud auth login

This will open your browser for authentication.

2.3 Set Application Default Credentials

gcloud auth application-default login

This allows Terraform to authenticate with GCP.

2.4 Set Your Project

List available projects:

gcloud projects list

Set the project you want to use:

gcloud config set project YOUR_PROJECT_ID

2.5 Verify Current Project

gcloud config get-value project

Step 3: Create Terraform Configuration Files

3.1 Create Project Directory

mkdir gcp-event-notification-setup
cd gcp-event-notification-setup

3.2 Create Main Terraform Configuration

Create a file named main.tf with the following content:

terraform {
  required_version = ">= 1.0"
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 5.0"
    }
  }
}

provider "google" {
  project = var.project_id
  region  = var.region
}

# Variables for customization
variable "project_id" {
  description = "GCP Project ID"
  type        = string
}

variable "region" {
  description = "GCP Region for resources"
  type        = string
  default     = "us-central1"
}

variable "service_account_name" {
  description = "Name for the service account"
  type        = string
  default     = "event-subscription-service"
}

variable "custom_role_id" {
  description = "ID for the custom IAM role"
  type        = string
  default     = "gcs_event_notification_manager"
}

variable "custom_role_title" {
  description = "Title for the custom IAM role"
  type        = string
  default     = "GCS Event Notification Manager"
}

# Enable required APIs
resource "google_project_service" "required_apis" {
  for_each = toset([
    "cloudresourcemanager.googleapis.com",
    "pubsub.googleapis.com",
    "storage.googleapis.com",
    "secretmanager.googleapis.com",
    "iam.googleapis.com",
    "iamcredentials.googleapis.com"
  ])

  project            = var.project_id
  service            = each.value
  disable_on_destroy = false
}

# Create Custom IAM Role
resource "google_project_iam_custom_role" "event_notification_manager" {
  role_id     = var.custom_role_id
  title       = var.custom_role_title
  description = "Allows management of Cloud Storage event notifications and Pub/Sub resources"
  project     = var.project_id
  stage       = "GA"

  permissions = [
    # Storage Permissions
    "storage.buckets.get",
    "storage.buckets.update",

    # Pub/Sub Topic Permissions
    "pubsub.topics.create",
    "pubsub.topics.delete",
    "pubsub.topics.get",
    "pubsub.topics.getIamPolicy",
    "pubsub.topics.list",
    "pubsub.topics.publish",
    "pubsub.topics.setIamPolicy",
    "pubsub.topics.attachSubscription",

    # Pub/Sub Subscription Permissions
    "pubsub.subscriptions.create",
    "pubsub.subscriptions.delete",
    "pubsub.subscriptions.get",
    "pubsub.subscriptions.getIamPolicy",
    "pubsub.subscriptions.list",
    "pubsub.subscriptions.setIamPolicy",
    "pubsub.subscriptions.update",
    "pubsub.subscriptions.consume",
    "pubsub.snapshots.seek",

    # Resource Manager Permissions
    "resourcemanager.projects.get",

    # IAM Permission
    "iam.serviceAccounts.actAs"
  ]

  depends_on = [google_project_service.required_apis]
}

# Create Service Account
resource "google_service_account" "event_subscription_service" {
  account_id   = var.service_account_name
  display_name = "Event Subscription Service Account"
  description  = "Service account for managing Cloud Storage event notifications"
  project      = var.project_id

  depends_on = [google_project_service.required_apis]
}

# Bind Custom Role to Service Account
resource "google_project_iam_member" "service_account_role_binding" {
  project = var.project_id
  role    = google_project_iam_custom_role.event_notification_manager.id
  member  = "serviceAccount:${google_service_account.event_subscription_service.email}"

  depends_on = [
    google_service_account.event_subscription_service,
    google_project_iam_custom_role.event_notification_manager
  ]
}

# Create Service Account Key
resource "google_service_account_key" "event_subscription_key" {
  service_account_id = google_service_account.event_subscription_service.name

  depends_on = [
    google_service_account.event_subscription_service,
    google_project_iam_member.service_account_role_binding
  ]
}

# Store Service Account Key in Secret Manager
resource "google_secret_manager_secret" "service_account_credentials" {
  secret_id = "${var.service_account_name}-credentials"
  project   = var.project_id

  replication {
    auto {}
  }

  labels = {
    purpose    = "gcs-event-subscription"
    managed-by = "terraform"
  }

  depends_on = [google_project_service.required_apis]
}

# Store the actual key content in the secret
resource "google_secret_manager_secret_version" "service_account_credentials_version" {
  secret      = google_secret_manager_secret.service_account_credentials.id
  secret_data = base64decode(google_service_account_key.event_subscription_key.private_key)

  depends_on = [google_secret_manager_secret.service_account_credentials]
}

# Grant the service account access to read its own credentials from Secret Manager
resource "google_secret_manager_secret_iam_member" "service_account_secret_accessor" {
  secret_id = google_secret_manager_secret.service_account_credentials.id
  role      = "roles/secretmanager.secretAccessor"
  member    = "serviceAccount:${google_service_account.event_subscription_service.email}"

  depends_on = [google_secret_manager_secret_version.service_account_credentials_version]
}

# Store credentials in local file (optional - for manual retrieval)
resource "local_file" "credentials" {
  content         = base64decode(google_service_account_key.event_subscription_key.private_key)
  filename        = "${path.module}/gcp-credentials.json"
  file_permission = "0600"
}

# Outputs
output "project_id" {
  description = "GCP Project ID"
  value       = var.project_id
}

output "region" {
  description = "GCP Region"
  value       = var.region
}

output "service_account_email" {
  description = "Email of the created service account"
  value       = google_service_account.event_subscription_service.email
}

output "service_account_id" {
  description = "ID of the created service account"
  value       = google_service_account.event_subscription_service.id
}

output "custom_role_id" {
  description = "ID of the custom IAM role"
  value       = google_project_iam_custom_role.event_notification_manager.id
}

output "custom_role_name" {
  description = "Name of the custom IAM role"
  value       = google_project_iam_custom_role.event_notification_manager.name
}

output "secret_manager_secret_id" {
  description = "Secret Manager secret ID containing service account credentials"
  value       = google_secret_manager_secret.service_account_credentials.secret_id
}

output "secret_manager_secret_name" {
  description = "Full Secret Manager secret name"
  value       = google_secret_manager_secret.service_account_credentials.name
}

output "credentials_file" {
  description = "Path to credentials file"
  value       = "Credentials saved to: ${abspath(local_file.credentials.filename)}"
}

output "retrieve_credentials_command" {
  description = "Command to retrieve service account credentials from Secret Manager"
  value       = "gcloud secrets versions access latest --secret=${google_secret_manager_secret.service_account_credentials.secret_id} --project=${var.project_id}"
}

output "service_account_key_download_note" {
  description = "Note about downloading service account key"
  value       = "Service account key is stored in Secret Manager. Use the retrieve_credentials_command to get the credentials."
}

3.3 Create Variables File

Create a file named terraform.tfvars with your values:

Option A: Default Configuration (Recommended)

project_id           = "your-project-id"
region               = "us-central1"
service_account_name = "event-subscription-service"

Option B: Custom Configuration

project_id           = "your-project-id"
region               = "us-east1"
service_account_name = "prod-event-service"
custom_role_id       = "custom_gcs_event_manager"
custom_role_title    = "Production Event Manager"

Replace your-project-id with your actual GCP project ID.

Step 4: Deploy Infrastructure

4.1 Initialize Terraform

terraform init

Expected output:

Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/google versions matching "~> 5.0"...
Terraform has been successfully initialized!

4.2 Preview Changes

terraform plan

This shows what will be created. Review carefully.

You should see:

6 APIs to be enabled
1 custom IAM role to be created
1 service account to be created
1 service account key to be generated
1 IAM role binding to be created
2 Secret Manager resources to be created
1 local file to be created

4.3 Apply Configuration

terraform apply

Type yes when prompted.

Deployment takes 2-3 minutes.

4.4 View Outputs

After successful deployment, you'll see outputs like:

Outputs:

credentials_file = "Credentials saved to: /path/to/gcp-event-notification-setup/gcp-credentials.json"
custom_role_id = "projects/your-project-id/roles/gcs_event_notification_manager"
custom_role_name = "projects/your-project-id/roles/gcs_event_notification_manager"
project_id = "your-project-id"
region = "us-central1"
retrieve_credentials_command = "gcloud secrets versions access latest --secret=event-subscription-service-credentials --project=your-project-id"
service_account_email = "event-subscription-service@your-project-id.iam.gserviceaccount.com"
service_account_id = "projects/your-project-id/serviceAccounts/event-subscription-service@your-project-id.iam.gserviceaccount.com"
service_account_key_download_note = "Service account key is stored in Secret Manager. Use the retrieve_credentials_command to get the credentials."
secret_manager_secret_id = "event-subscription-service-credentials"
secret_manager_secret_name = "projects/123456789/secrets/event-subscription-service-credentials"

Step 5: Retrieve and Use Credentials

5.1 Get Credentials from Local File

The credentials are automatically saved to a local file:

cat gcp-credentials.json

5.2 Get Credentials from Secret Manager

Retrieve from Secret Manager (recommended for production):

gcloud secrets versions access latest \
  --secret=event-subscription-service-credentials \
  --project=your-project-id > retrieved-credentials.json

5.3 Verify Service Account

Check the service account was created:

gcloud iam service-accounts list --project=your-project-id

You should see:

NAME                          EMAIL
Event Subscription Service Account   event-subscription-service@your-project-id.iam.gserviceaccount.com

5.4 Verify Custom Role

Check the custom role:

gcloud iam roles describe gcs_event_notification_manager --project=your-project-id

5.5 Verify Role Binding

Check IAM policy:

gcloud projects get-iam-policy your-project-id \
  --flatten="bindings[].members" \
  --filter="bindings.members:serviceAccount:event-subscription-service@*"

Step 6: Verify Deployment

6.1 View All Resources Created

terraform state list

Expected output:

google_project_iam_custom_role.event_notification_manager
google_project_iam_member.service_account_role_binding
google_project_service.required_apis["cloudresourcemanager.googleapis.com"]
google_project_service.required_apis["iam.googleapis.com"]
google_project_service.required_apis["iamcredentials.googleapis.com"]
google_project_service.required_apis["pubsub.googleapis.com"]
google_project_service.required_apis["secretmanager.googleapis.com"]
google_project_service.required_apis["storage.googleapis.com"]
google_secret_manager_secret.service_account_credentials
google_secret_manager_secret_iam_member.service_account_secret_accessor
google_secret_manager_secret_version.service_account_credentials_version
google_service_account.event_subscription_service
google_service_account_key.event_subscription_key
local_file.credentials

6.2 Verify Service Account

Check the service account was created:

gcloud iam service-accounts list --project=your-project-id

You should see:

NAME                               EMAIL
Event Subscription Service Account event-subscription-service@your-project-id.iam.gserviceaccount.com

6.3 Verify Custom Role

Check the custom role:

gcloud iam roles describe gcs_event_notification_manager --project=your-project-id

6.4 Verify IAM Policy Binding

Check the role assignment:

gcloud projects get-iam-policy your-project-id \
  --flatten="bindings[].members" \
  --filter="bindings.members:serviceAccount:event-subscription-service@*"

6.5 Verify Enabled APIs

gcloud services list --enabled --project=your-project-id

Step 7: Retrieve Service Account Credentials

7.1 Option 1: From Local File

The credentials are automatically saved to a local file:

cat gcp-credentials.json

7.2 Option 2: From Secret Manager

Retrieve from Secret Manager (recommended for production):

gcloud secrets versions access latest \
  --secret=event-subscription-service-credentials \
  --project=your-project-id > retrieved-credentials.json

7.3 Option 3: From Terraform Output

View the Terraform output that shows where credentials are stored:

terraform output credentials_file
terraform output retrieve_credentials_command

Last Updated: November 2025
Version: 1.0
Author: GCP Event Notification Setup Team

Prerequisites​

Step 1: Install Required Tools​

1.1 Install gcloud CLI​

1.2 Install Terraform​

Step 2: Authenticate with GCP​

2.1 Initialize gcloud​

2.2 Login to GCP​

2.3 Set Application Default Credentials​

2.4 Set Your Project​

2.5 Verify Current Project​

Step 3: Create Terraform Configuration Files​

3.1 Create Project Directory​

3.2 Create Main Terraform Configuration​

3.3 Create Variables File​

Step 4: Deploy Infrastructure​

4.1 Initialize Terraform​

4.2 Preview Changes​

4.3 Apply Configuration​

4.4 View Outputs​

Step 5: Retrieve and Use Credentials​

5.1 Get Credentials from Local File​

5.2 Get Credentials from Secret Manager​

5.3 Verify Service Account​

5.4 Verify Custom Role​

5.5 Verify Role Binding​

Step 6: Verify Deployment​

6.1 View All Resources Created​

6.2 Verify Service Account​

6.3 Verify Custom Role​

6.4 Verify IAM Policy Binding​

6.5 Verify Enabled APIs​

Step 7: Retrieve Service Account Credentials​

7.1 Option 1: From Local File​

7.2 Option 2: From Secret Manager​

7.3 Option 3: From Terraform Output​

Prerequisites

Step 1: Install Required Tools

1.1 Install gcloud CLI

1.2 Install Terraform

Step 2: Authenticate with GCP

2.1 Initialize gcloud

2.2 Login to GCP

2.3 Set Application Default Credentials

2.4 Set Your Project

2.5 Verify Current Project

Step 3: Create Terraform Configuration Files

3.1 Create Project Directory

3.2 Create Main Terraform Configuration

3.3 Create Variables File

Step 4: Deploy Infrastructure

4.1 Initialize Terraform

4.2 Preview Changes

4.3 Apply Configuration

4.4 View Outputs

Step 5: Retrieve and Use Credentials

5.1 Get Credentials from Local File

5.2 Get Credentials from Secret Manager

5.3 Verify Service Account

5.4 Verify Custom Role

5.5 Verify Role Binding

Step 6: Verify Deployment

6.1 View All Resources Created

6.2 Verify Service Account

6.3 Verify Custom Role

6.4 Verify IAM Policy Binding

6.5 Verify Enabled APIs

Step 7: Retrieve Service Account Credentials

7.1 Option 1: From Local File

7.2 Option 2: From Secret Manager

7.3 Option 3: From Terraform Output