Google Cloud Storage:Notification 實作 (一) – 使用於 Cloud Functions

GCS Notification 功能實作 (一):應用於 Cloud Functions

前置準備

1. 開啟 Cloud functions & Cloud storage API
2. 確保有安裝 Cloud SDK
3. 確保有 Node.js & nvm 開發環境

建立 Bucket 與測試資料

1. 創立一個 Cloud Storage Bucket, [Bucket名稱] = 自訂的bucket

$ gsutil mb gs://[Bucket名稱]

2. 創立並移動至目錄夾

$ mkdir ~/gcf_gcs
$ cd ~/gcf_gcs

3. 在 gcf_gcs 的目錄下建立 index.js 並貼入以下的程式

$ touch index.js
$ vim index.js

(以下為 javascript)

exports.helloGCSGeneric = (event, callback) => {
 const file = event.data;
 const context = event.context;

 console.log(`Event ${context.eventId}`);
 console.log(`  Event Type: ${context.eventType}`);
 console.log(`  Bucket: ${file.bucket}`);
 console.log(`  File: ${file.name}`);
 console.log(`  Metageneration: ${file.metageneration}`);
 console.log(`  Created: ${file.timeCreated}`);
 console.log(`  Updated: ${file.updated}`);

 callback();
};

Object Finalize Event (更新動作)

1. 這個功能在有新的 object 被寫入或是更新 bucket 時會被觸發:
2. 在 gcf_gcs 的目錄下部署此功能

$ gcloud beta functions deploy helloGCSGeneric --trigger-resource [Bucket名稱] --trigger-event google.storage.object.finalize

3. 去觸發此功能

$ touch gcf-test.txt // 新增檔案
$ gsutil cp gcf-test.txt gs://[Bucket名稱] // 複製檔案到Bucket
$ gcloud beta functions logs read --limit 50 // 查看log

4. 成功執行會看到最下方有剛剛複製檔案到 cloud storage 的 log

I      helloGCSGeneric  72659071387910 2018-04-09 08:06:35.589    Bucket: [Bucket名稱]
I      helloGCSGeneric  72659071387910 2018-04-09 08:06:35.589    File: gcf-test.txt
I      helloGCSGeneric  72659071387910 2018-04-09 08:06:35.589    Metageneration: 1
I      helloGCSGeneric  72659071387910 2018-04-09 08:06:35.589    Created: 2018-04-09T08:06:33.656Z
I      helloGCSGeneric  72659071387910 2018-04-09 08:06:35.589    Updated: 2018-04-09T08:06:33.656Z

Object Delete Event (刪除動作)

1. 部署此功能

$ gcloud beta functions deploy helloGCSGeneric --trigger-resource [Bucket名稱] --trigger-event google.storage.object.delete

2. 新增一個檔案

$ touch gcf-delete.txt

3. 確保bucket 是 non-versioning (object versioning: 當此功能被開啟時,每次有修改或是刪除動作時,Cloud Storage會自動對目前bucket的內容形成一個unique的檔案)

$ gsutil versioning set off gs://[Bucket名稱]

4. 上傳 gcf-delete.txt 檔案到bucket 並刪除檔案

$ gsutil cp gcf-delete.txt gs://[Bucket名稱]   //上傳
$ gsutil rm gs://[Bucket名稱]/gcf-delete.txt   //刪除
$ gcloud beta functions logs read --limit 50   // 查看log

5. 查看log 是否有確實刪除檔案

D      helloGCSGeneric  72660870238980 2018-04-09 09:02:49.546  Function execution started
I      helloGCSGeneric  72660870238980 2018-04-09 09:02:49.780  Event 72660870238980
I      helloGCSGeneric  72660870238980 2018-04-09 09:02:49.788    Event Type: google.storage.object.delete
I      helloGCSGeneric  72660870238980 2018-04-09 09:02:49.788    Bucket: [Bucket名稱]
I      helloGCSGeneric  72660870238980 2018-04-09 09:02:49.788    File: gcf-delete.txt
I      helloGCSGeneric  72660870238980 2018-04-09 09:02:49.788    Metageneration: 1
I      helloGCSGeneric  72660870238980 2018-04-09 09:02:49.788    Created: 2018-04-09T09:02:27.332Z
I      helloGCSGeneric  72660870238980 2018-04-09 09:02:49.788    Updated: 2018-04-09T09:02:27.332Z
D      helloGCSGeneric  72660870238980 2018-04-09 09:02:49.882  Function execution took 337 ms, finished with status: 'ok'

Object Archive Event (歸檔動作)

1. 此功能只能執行在 versioning bucket 上。當一個舊版本的 Object 被 archive,也就是當一個 Object 被修改或是刪除時,Archive event 會被觸發。
2. 建立一個 index.js 檔案並且貼上與上方相同的程式碼
3. 部署此功能

$ gcloud beta functions deploy helloGCSGeneric --trigger-resource [Bucket名稱] --trigger-event google.storage.object.archive

1. 新增一個檔案

$ touch gcf-archive.txt

2. 確保bucket 是 versioning

$ gsutil versioning set on gs://[Bucket名稱]

3. 上傳 gcf-archive.txt 檔案到 bucket 並刪除檔案去觸發 archive

$ gsutil cp gcf-archive.txt gs://[Bucket名稱]   //上傳
$ gsutil rm gs://[Bucket名稱]/gcf-archive.txt   //刪除
$ gcloud beta functions logs read --limit 50   // 查看log

4. 查看 log 是否有確實刪除檔案去觸發 archive

D      helloGCSGeneric  72691716149423 2018-04-09 09:27:40.205  Function execution started
I      helloGCSGeneric  72691716149423 2018-04-09 09:27:40.410  Event 72691716149423
I      helloGCSGeneric  72691716149423 2018-04-09 09:27:40.422    Event Type: google.storage.object.archive
I      helloGCSGeneric  72691716149423 2018-04-09 09:27:40.422    Bucket: [Bucket名稱]I helloGCSGeneric 72691716149423  2018-04-09 09:27:40.422 File: gcf-archive.txt
I      helloGCSGeneric  72691716149423 2018-04-09 09:27:40.422    Metageneration: 1
I      helloGCSGeneric  72691716149423 2018-04-09 09:27:40.422    Created: 2018-04-09T09:27:08.460Z
I      helloGCSGeneric  72691716149423 2018-04-09 09:27:40.422    Updated: 2018-04-09T09:27:08.460Z
D      helloGCSGeneric  72691716149423 2018-04-09 09:27:40.516  Function execution took 376 ms, finished with status: 'ok'

Object Metadata Update

1. 此功能在 metadata object 被修改時會被觸發
2. 建立一個 index.js 檔案並且貼上與上方相同的程式碼
3. 部署此功能

$ gcloud beta functions deploy helloGCSGeneric --trigger-resource [Bucket名稱] --trigger-event google.storage.object.metadataUpdate

4. 新增一個檔案

$ touch gcf-metadata.txt

5. 確保 bucket 是 non-versioning

$ gsutil versioning set off gs://[Bucket名稱]

6. 上傳檔案並更新 metadata

$ gsutil cp gcf-metadata.txt gs://[Bucket名稱]
$ gsutil -m setmeta -h "Content-Type:text/plain" gs://[Bucket名稱]/gcf-metadata.txt
$ gcloud beta functions logs read --limit 50   // 查看log

7. 查看log

D      helloGCSGeneric  70171271580852 2018-04-09 09:47:06.947  Function execution started
I      helloGCSGeneric  70171271580852 2018-04-09 09:47:07.305  Event 70171271580852
I      helloGCSGeneric  70171271580852 2018-04-09 09:47:07.313    Event Type: google.storage.object.metadataUpdate
I      helloGCSGeneric  70171271580852 2018-04-09 09:47:07.313    Bucket: gcp-expert-csbucket
I      helloGCSGeneric  70171271580852 2018-04-09 09:47:07.313    File: gcf-metadata.txt
I      helloGCSGeneric  70171271580852 2018-04-09 09:47:07.313    Metageneration: 2
I      helloGCSGeneric  70171271580852 2018-04-09 09:47:07.313    Created: 2018-04-09T09:46:56.932Z
I      helloGCSGeneric  70171271580852 2018-04-09 09:47:07.313    Updated: 2018-04-09T09:47:05.567Z
D      helloGCSGeneric  70171271580852 2018-04-09 09:47:07.414  Function execution took 467 ms, finished with status: 'ok'

參考資料:https://cloud.google.com/functions/docs/tutorials/storage


連絡「GCP 專門家」