Cross-region DataGuard in OCI coded with Terraform

In one of the latest lessons of my tutorial I have shown how to setup DataGuard locally within the same region. I guess this is some sort of database protection, but in case of the whole region failure it will not help at all. So it is worth to consider the cross-region solution for database security. In this additional lesson I am trying to showcase how to deploy remote DataGuard Association with the usage of Terraform code.

Pic. Topology Diagram

The database archivelogs will be send directly via DRGs and OCI backbone. I hope you will find this dish tasty and it will be somehow useful for your work with the cloud solutions. Let me know if you have any kind of feedback. 🙂

Bon Appetit,

Martin, The Cook.

Subscribe to get access

For now video is blocked, but you can watch it immediately when you subscribe!

Feedback loop.

linkedin_groupThe last two months, during the COVID thing, were transformational for this blog. Someone can say the window of the opportunity has been opened by free training and certifications for OCI. Many of us have decided to ride this horse and discover OCI, in-depth. Some of us have reached the highest professional level, some of us are still on the associate one. No matter if you plan to be OCI Architect, OCI Developer, or maybe OCI Ops, there is a lot of work in front of us. It will be a permanent learning cycle.  This is characteristic of the modern cloud world. New features, new services, new knowledge to be discovered and explored. Frankly speaking sometimes I feel overwhelmed, by this massive amount of “new stuff”. I believe you have this feeling too. And here comes the community. That is why I have built a community group on LinkedIn to exchange knowledge. Someone can say it is just another channel to promote this blog, but believe me – my goal has been simple, build the place to exchange knowledge. As much as possible … people who are satisfied with answers… people able to find some form of support from the others. A feedback loop is crucial. Without a feedback loop,  the community doesn’t exist. So I really encourage everyone to ask questions and the others to share the experience in response.

BTW. Many of you are asking how to help, support our blog, YT channel, and LinkedIn group. You can join us to be the next cook in the kitchen (don’t be afraid, this kitchen is extremely big and there is a lot of space to work). Another option is to donate us (check up on the right). If you want to be just a reader of the blog or watcher on YT, I think it is  absolutely OK 🙂 But remember about the feedback loop. If you haven’t subscribed to the channel, please do it now. You can also give us feedback on Linkedin with your likes 🙂

Best,

Martin, The Cook.

How to build microservice which is based on OCI Function and ATP with Terraform

A lot of people around the world believe the serverless approach is the ultimate future of cloud computing. I am also such a believer, but it will be a gradual move, sort of evolution within the next couple of years. Some time ago I have published a blog about private endpoints for ATP.

Pic. Topology Diagram

I have used the code related to this blog post and I have refactored it a bit. The idea behind was to eliminate VM-based web server with Flask microservice. This old school technology could be replaced by OCI Function and API Gateway (btw. check Kranthi blog post). I guess this refactoring has been pretty successful. Below in this 15 minutes video, you will see my hard work on this topic. Hope it will be tasty 🙂

Bon Appetit,

Martin, The Cook.

Subscribe to get access

For now video is blocked, but you can watch it immediately when you subscribe!

Terraform state file management

Following up on my previous post about storing the terraform state file on a remote object storage location, I would like to talk about the terraform state command

The terraform state command is used for advanced state management. As your Terraform usage becomes more advanced, there are some cases where you may need to modify the Terraform state. Rather than modify the state directly, the terraform state commands can be used in many cases instead.

A typical use case that requires modifying an existing state file is refactoring of existing code (e.g upgrading your code from terraform 0.11x to 0.12x and/or moving from the count construct to the new for_each construct)

Let’s consider this example:

variable “list” {
default = [“linus”, “cheetarah”, “li-shou”]
}

resource random_pet “zoo” {
count = 3
prefix = var.list[count.index]
}

 

 

It’s a simple test case that makes use of the random provider: we define a list of three items, and then use the count construct to deploy all of them with a single resource definition. This was the only way to deploy multiple instances of a resource in terraform 0.11

mattiarossi@imacpro-01 ~/D/C/C/TEST> terraform plan
Refreshing Terraform state in-memory prior to plan…
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
————————————————————————

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create

Terraform will perform the following actions:

# random_pet.zoo[0] will be created
+ resource “random_pet” “zoo” {
+ id = (known after apply)
+ length = 2
+ prefix = “linus”
+ separator = “-”
}

# random_pet.zoo[1] will be created
+ resource “random_pet” “zoo” {
+ id = (known after apply)
+ length = 2
+ prefix = “cheetarah”
+ separator = “-”
}

# random_pet.zoo[2] will be created
+ resource “random_pet” “zoo” {
+ id = (known after apply)
+ length = 2
+ prefix = “li-shou”
+ separator = “-”
}

Plan: 3 to add, 0 to change, 0 to destroy.

————————————————————————

Note: You didn’t specify an “-out” parameter to save this plan, so Terraform
can’t guarantee that exactly these actions will be performed if
“terraform apply” is subsequently run.

 

If we apply this plan, the result will be:

mattiarossi@imacpro-01 ~/D/C/C/TEST> terraform apply

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create

Terraform will perform the following actions:

# random_pet.zoo[0] will be created
+ resource “random_pet” “zoo” {
+ id = (known after apply)
+ length = 2
+ prefix = “linus”
+ separator = “-”
}

# random_pet.zoo[1] will be created
+ resource “random_pet” “zoo” {
+ id = (known after apply)
+ length = 2
+ prefix = “cheetarah”
+ separator = “-”
}

# random_pet.zoo[2] will be created
+ resource “random_pet” “zoo” {
+ id = (known after apply)
+ length = 2
+ prefix = “li-shou”
+ separator = “-”
}

Plan: 3 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
Terraform will perform the actions described above.
Only ‘yes’ will be accepted to approve.

Enter a value: yes

random_pet.zoo[0]: Creating…
random_pet.zoo[1]: Creating…
random_pet.zoo[2]: Creating…
random_pet.zoo[1]: Creation complete after 0s [id=cheetarah-magnetic-woodcock]
random_pet.zoo[2]: Creation complete after 0s [id=li-shou-immense-gnu]
random_pet.zoo[0]: Creation complete after 0s [id=linus-settled-bluegill]

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

 

and our state file will contain:

mattiarossi@imacpro-01 ~/D/C/C/TEST> cat terraform.tfstate
{
“version”: 4,
“terraform_version”: “0.12.24”,
“serial”: 1,
“lineage”: “18b586d4-7da1-5dea-8aa8-d78bef761e7d”,
“outputs”: {},
“resources”: [
{
“mode”: “managed”,
“type”: “random_pet”,
“name”: “zoo”,
“each”: “list”,
“provider”: “provider.random”,
“instances”: [
{
“index_key”: 0,
“schema_version”: 0,
“attributes”: {
“id”: “linus-settled-bluegill”,
“keepers”: null,
“length”: 2,
“prefix”: “linus”,
“separator”: “-”
},
“private”: “bnVsbA==”
},
{
“index_key”: 1,
“schema_version”: 0,
“attributes”: {
“id”: “cheetarah-magnetic-woodcock”,
“keepers”: null,
“length”: 2,
“prefix”: “cheetarah”,
“separator”: “-”
},
“private”: “bnVsbA==”
},
{
“index_key”: 2,
“schema_version”: 0,
“attributes”: {
“id”: “li-shou-immense-gnu”,
“keepers”: null,
“length”: 2,
“prefix”: “li-shou”,
“separator”: “-”
},
“private”: “bnVsbA==”
variable “list” {
}
]
}
]
}

Now we have a state file we can query:

mattiarossi@imacpro-01 ~/D/C/C/TEST> terraform state list
random_pet.zoo[0]
random_pet.zoo[1]
random_pet.zoo[2]

mattiarossi@imacpro-01 ~/D/C/C/TEST> terraform state show random_pet.zoo[0]
# random_pet.zoo[0]:
resource “random_pet” “zoo” {
id = “linus-settled-bluegill”
length = 2
prefix = “linus”
separator = “-”
}

This is an oversimplified example, but let’s say that because of a refactoring we need to move our resource definition from using the count construct to the for_each construct:

variable “list” {
default = [“linus”, “cheetarah”, “li-shou”]
}

resource random_pet “for_each_zoo” {
for_each = toset(var.list)
prefix = each.key
}

If we run a plan, it will say we need to delete the old resources and create three new ones, even if logically they are the same. In a real world use case this would have happened to resources like VMs, attached and configured block volumes and/or network interface, or object storage buckets, and we really wouldn’t want to destroy and recreate them, especially because of an infrastructure code refactor.

 

mattiarossi@imacpro-01 ~/D/C/C/TEST> terraform plan
Refreshing Terraform state in-memory prior to plan…
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

random_pet.zoo[1]: Refreshing state… [id=cheetarah-magnetic-woodcock]
random_pet.zoo[0]: Refreshing state… [id=linus-settled-bluegill]
random_pet.zoo[2]: Refreshing state… [id=li-shou-immense-gnu]

————————————————————————

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
– destroy

Terraform will perform the following actions:

# random_pet.for_each_zoo[“cheetarah”] will be created
+ resource “random_pet” “for_each_zoo” {
+ id = (known after apply)
+ length = 2
+ prefix = “cheetarah”
+ separator = “-”
}

# random_pet.for_each_zoo[“li-shou”] will be created
+ resource “random_pet” “for_each_zoo” {
+ id = (known after apply)
+ length = 2
+ prefix = “li-shou”
+ separator = “-”
}

# random_pet.for_each_zoo[“linus”] will be created
+ resource “random_pet” “for_each_zoo” {
+ id = (known after apply)
+ length = 2
+ prefix = “linus”
+ separator = “-”
}

# random_pet.zoo[0] will be destroyed
– resource “random_pet” “zoo” {
– id = “linus-settled-bluegill” -> null
– length = 2 -> null
– prefix = “linus” -> null
– separator = “-” -> null
}

# random_pet.zoo[1] will be destroyed
– resource “random_pet” “zoo” {
– id = “cheetarah-magnetic-woodcock” -> null
– length = 2 -> null
– prefix = “cheetarah” -> null
– separator = “-” -> null
}

# random_pet.zoo[2] will be destroyed
– resource “random_pet” “zoo” {
– id = “li-shou-immense-gnu” -> null
– length = 2 -> null
– prefix = “li-shou” -> null
– separator = “-” -> null
}

Plan: 3 to add, 0 to change, 3 to destroy.

————————————————————————

 

What the terraform state command allow us to do is to ‘move around’ resources so that they will not be considered as a change in the plan:

mattiarossi@imacpro-01 ~/D/C/C/TEST> terraform state mv ‘random_pet.zoo[1]’ ‘random_pet.for_each_zoo[“cheetarah”]’
Move “random_pet.zoo[1]” to “random_pet.for_each_zoo[\”cheetarah\”]”
Successfully moved 1 object(s).
mattiarossi@imacpro-01 ~/D/C/C/TEST> terraform state mv ‘random_pet.zoo[2]’ ‘random_pet.for_each_zoo[“li-shou”]’
Move “random_pet.zoo[2]” to “random_pet.for_each_zoo[\”li-shou\”]”
Successfully moved 1 object(s).
mattiarossi@imacpro-01 ~/D/C/C/TEST> terraform state mv ‘random_pet.zoo[0]’ ‘random_pet.for_each_zoo[“linus”]’
Move “random_pet.zoo[0]” to “random_pet.for_each_zoo[\”linus\”]”
Successfully moved 1 object(s).

mattiarossi@imacpro-01 ~/D/C/C/TEST> terraform state list
random_pet.for_each_zoo[“cheetarah”]
random_pet.for_each_zoo[“li-shou”]
random_pet.for_each_zoo[“linus”]

 

If we run our plan after the move, there will be no changes to apply:

mattiarossi@imacpro-01 ~/D/C/C/TEST> terraform plan
Refreshing Terraform state in-memory prior to plan…
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

random_pet.for_each_zoo[“linus”]: Refreshing state… [id=linus-settled-bluegill]
random_pet.for_each_zoo[“cheetarah”]: Refreshing state… [id=cheetarah-magnetic-woodcock]
random_pet.for_each_zoo[“li-shou”]: Refreshing state… [id=li-shou-immense-gnu]

————————————————————————

No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.

This new terraform functionality provides a lot of flexibility when upgrading/updating infrastructure code, as the state management can be programmatically scripted as well.

Previous to this a refactoring would mean manually changing the json state file, with the risk of ending up with an unusable file.

Have fun!

Mattia

 

How to use OCI Marketplace image with Terraform?

Recently Mattia has presented the blog post about using OCI CLI to retrieve OCI Marketplace image. It has inspired me to explore this topic, but with Terraform usage. The goal was very simple – deploy RocketChat image (from OCI Marketplace) as a single VM in the extremely simplified cloud infrastructure, very similar to my first lesson of OCI + Terraform tutorial. I have invested 3 hours and here is the outcome of this work. I hope you will find it tasty, despite the fact it is a really tricky setup, with many dependent resources and data sources.

Bon Appetit,

Martin, The Cook.

 

How to retrieve an OCI Marketplace image id using advanced functionality in the OCI CLI utility

If you ever need to deploy an OCI Marketplace image using your automation, the information that you will have to feed to your script will be:

  "listingId": "ocid1.appcataloglisting.oc1..aaaaaaaam7ewzrjbltqiarxukuk72v2lqkdtpqtwxqpszqqvrm7likfnpt5q",
  "listingResourceId": "ocid1.image.oc1..aaaaaaaaa5u7qtj3j3um4zryg3qkn6if2sqctqjfh46pdq5z56kq6zagg4va",

Let’s say we need to find this information for a specific Marketplace image, like RocketChat:

Screenshot 2020-05-19 at 15.04.55

We can use the OCI cli to retrieve the details needed by terraform, but since there is no direct command to fetch information about a Marketplace listing we are going to use the ‘raw-request’ functionality that the cli provides as a fallback to access the OCI API when needed for advanced use cases

I have created a gist that uses the OCI cli to perform this kind of search, you can find it on GitHub

#!/bin/bash 
#Example string: 'FortiGate Next-Gen Firewall (BYOL)'
export STERM="$1"
#Example version: '6.4.0'
export SVERSION="$2" 
export REGION="uk-london-1" 
export LID=`oci raw-request --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings --http-method GET | jq --arg STERM "$STERM" '.data[] | select ( .displayName | contains($STERM))' | jq -r '.listingId'` 
oci raw-request --http-method GET --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings/$LID/resourceVersions | jq --arg SVERSION "$SVERSION" '.data[] | select ( .listingResourceVersion | contains($SVERSION))'

The script accepts two parameters

  • the Marketplace image name
  • the Image version needed
mrossi@cloudshell:~ (us-phoenix-1)$ ./getMarketplaceImageId 'RocketChat' '2.4.9'
{
"listingId": "ocid1.appcataloglisting.oc1..aaaaaaaapjvcl7yie67z6ag4hwfgpiegwpy5izkaaq3p3rngvmtmqm7zcaxq",
"listingResourceId": "ocid1.image.oc1..aaaaaaaajq2pfkmd6nl23ouho2ulxldoy2or5zy7dbiv46qhu6tsqm6ftila",
"listingResourceVersion": "2.4.9",
"timePublished": "2020-03-05T13:27:11.940Z"
}

Let’s break down the script:

export STERM="$1"
#6.4.0
export SVERSION="$2" 
export REGION="uk-london-1"

These are initialisation parameters, assigning the search parameters from the script input params, and setting a default OCI region for the API queries

export LID=`oci raw-request --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings --http-method GET | jq --arg STERM "$STERM" '.data[] | select ( .displayName | contains($STERM))' | jq -r '.listingId'`

The first call to the OCI cli retrieves the Listing id for a given Marketplace image:

mrossi@cloudshell:~ (us-phoenix-1)$ oci raw-request --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings --http-method GET | jq --arg STERM "$STERM" '.data[] | select ( .displayName | contains($STERM))' | jq -r '.listingId'
ocid1.appcataloglisting.oc1..aaaaaaaapjvcl7yie67z6ag4hwfgpiegwpy5izkaaq3p3rngvmtmqm7zcaxq

Let’s break down the command:

The first part retrieves all the Marketplace listings for a given region:

mrossi@cloudshell:~ (us-phoenix-1)$ oci raw-request --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings --http-method GET 
{
  "data": [
    {
      "displayName": "0Chain Data Privacy Compliance Platform",
      "listingId": "ocid1.appcataloglisting.oc1..aaaaaaaahfwb7mtjpi45wc4waduujja5fdeslfio4r2amuyuz3x62x5rxina",
      "publisherName": "0Chain LLC",
      "summary": "Add data privacy, protection, and private sharing to your business"
    },
....

    {
      "displayName": "RocketChat",
      "listingId": "ocid1.appcataloglisting.oc1..aaaaaaaapjvcl7yie67z6ag4hwfgpiegwpy5izkaaq3p3rngvmtmqm7zcaxq",
      "publisherName": "Rocket.Chat",
      "summary": "The ultimate Open Source communication platform"
    },
    {
      "displayName": "SUSE Linux Enterprise Server 12 SP4 (BYOS)",
      "listingId": "ocid1.appcataloglisting.oc1..aaaaaaaa3mflvqev6iw4jnjibtrlgdz2h75kpmdwgvhifljffv5b5xbcqmxq",
      "publisherName": "SUSE",
      "summary": "SLES 12 SP4"
    },
    {
      "displayName": "SUSE Linux Enterprise Server 12 SP4 (BYOS)",
      "listingId": "ocid1.appcataloglisting.oc1..aaaaaaaaowy672zei2w2dy3s2z4c7kytbrd55cnvox3djurgswguiang32ga",
      "publisherName": "SUSE",
      "summary": "SLES 12 SP4"
    },
 ...
  ],
  "headers": {
    "Connection": "keep-alive",
    "Content-Type": "application/json",
    "Date": "Tue, 19 May 2020 13:31:54 GMT",
    "Transfer-Encoding": "chunked",
    "Vary": "Accept-Encoding",
    "X-Content-Type-Options": "nosniff",
    "opc-next-page": "AAAAAAAAAAAAFCCBIFAUCQKBIFAUCQKHOVUVIY2NOFPTETJNMV5HQX3KKJKHE2DOHBZUWRTRNMZWWODWJIYHM3BTOE3VSVLTNJEGUSCPIVGVM5TGOM2GC3LCFVDHMTLCLA2XA6SXJVCEMRSNNNIHQVCPLFVGSU3DIVIGKTTZI5UE24LUI5KHQLKFOFVVKVLNO5UWS2KQOVWGM4LCOA3HQSLXOI4FATJWLFITAM2DIEZDK3KXMN3C233IJFLEGZZVNQYVATKZJAWUY5DPGFIUUWBVHBZWKSTOOFPUIUKGO5VTCSRZINBVE4JNKI4HKTTZN5SFO4CJOBAUUTDUKVSDKUCXNVVV6ZBTIZMVGYSVKFEHC4BYNJUHQOJQMNGWI6CNGAZFMSTJMFMEUQSCNV3VEZ32NFGWQVCGJFYDOLJTJBXUCVKYM5JWYYKGPFDWC5BSGF2HIZ27KFIEMXZQKVDEUUD2KRHGUVKNJNTFETKHIZGDEWTYGMYHG23CJNSGGMCEINXFKY2XJJWVG4JRGBKGOWSIKFUXKSCCKM3XAZ2EIJHWETLOPFXU23KRNA4UMNLOIQ2FU53CKRUEKWSYGFPUINSVJRWW63DRKVKUKQZVIF5FMSKMOZXG6WS7N5YF62LQMEZE4MLLLBRWQUCDOZUXA23WMJQTGODRJJHG62ZSKZ5HO4ZWGFKWQSLWHFSFQYLCMQYFINRUJRGVAWBWJA3UEVCYKAYFQ6TTJNYVKU2BGI4WSQLXMJDFUV2GPBLGIMDZM5QWY2KMNJTXQ6LCNBPWSSDUOIZE643IL5VG6SD2HBLDKSS2JVVHEVLULBQVE6TMOZWWSN3YJFKWI2TXMRQXQMLWJ5IXCNDSI5LS233SJI4WKSJXIRGTS2RQHFBXGNBVINRWUOCWLFBG26BVGVEECMZZIRDDQYLGJN3FS3DUNQYWGWLLNFHGYVTRONNFK2K2PF2XGT2JMRVWSZ3LG5LFCY2HORMXOODXOREEO3ZZLJVVU6CIKNUGK42DKJLEQTCMKEYXC3DFPBAWMQSNKZXTE5ZTFVHQ",
    "opc-request-id": "843C1ADDCBC74192AFA0499764567CD8/3349DFC98814A8F5F33B1F767B7A0FE6/7D4C870DF934CD46CD94537C86F0969D"
  },
  "status": "200 OK"
}

The second part filters the global result json object and retrieves only information related to the Image we’re interested in:

oci raw-request --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings --http-method GET | jq --arg STERM "$STERM" '.data[] | select ( .displayName | contains($STERM))'

I am using the jq utility to do that by setting up a filter on the displayName field:

jq --arg STERM "$STERM" '.data[] | select ( .displayName | contains($STERM))'
mrossi@cloudshell:~ (us-phoenix-1)$ oci raw-request --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings --http-method GET | jq --arg STERM "$STERM" '.data[] | select ( .displayName | contains($STERM))'
{
"displayName": "RocketChat",
"listingId": "ocid1.appcataloglisting.oc1..aaaaaaaapjvcl7yie67z6ag4hwfgpiegwpy5izkaaq3p3rngvmtmqm7zcaxq",
"publisherName": "Rocket.Chat",
"summary": "The ultimate Open Source communication platform"
}

The third part filters the single listing result json object and retrieves the listingId field, again using the jq utility and assigns it to an utility variable

export LID=`oci raw-request --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings --http-method GET | jq --arg STERM "$STERM" '.data[] | select ( .displayName | contains($STERM))' | jq -r '.listingId'`

Using the collected listing ID we can retrieve the actual information we’re interested in by issuing another API call:

oci raw-request --http-method GET --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings/$LID/resourceVersions | jq --arg SVERSION "$SVERSION" '.data[] | select ( .listingResourceVersion | contains($SVERSION))'

This command filters the API call output using jq to filter only the specific information we’re interested in:

mrossi@cloudshell:~ (us-phoenix-1)$ oci raw-request --http-method GET --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings/$LID/resourceVersions | jq --arg SVERSION "$SVERSION" '.data[] | select ( .listingResourceVersion | contains($SVERSION))'
{
"listingId": "ocid1.appcataloglisting.oc1..aaaaaaaapjvcl7yie67z6ag4hwfgpiegwpy5izkaaq3p3rngvmtmqm7zcaxq",
"listingResourceId": "ocid1.image.oc1..aaaaaaaajq2pfkmd6nl23ouho2ulxldoy2or5zy7dbiv46qhu6tsqm6ftila",
"listingResourceVersion": "2.4.9",
"timePublished": "2020-03-05T13:27:11.940Z"
}

This technique of doing RAW api calls can be extended for any use that the standard cli commands do not cover, while the filtering through jq can be used for manipulating any json output. The OCI cli has an internal filtering mechanism that doesn’t play well with the output from the raw request subcommand

Have fun !

Mattia

My thoughts after successful Oracle Cloud Infrastructure Developer 2020 Certified Associate exam

mattia_rossi_oci_certified_solution_architect_associate_2018Last week I was very busy so it was hard to find a few free cycles to write about it. Yes, I have passed recently Oracle Cloud Infrastructure Developer 2020 Certified Associate exam. Frankly speaking, it was obvious for me I need to do it. I am a hardcore developer here at FoggyKitchen, mostly with Terraform, but I am also writing a lot of stuff in Ansible for Oracle, for example, Ansible-Database-Migration-Tool (ADMT). You are probably curious if this certification is important and if it is worth spending time on it. Well… I guess it is. The modern world is changing like crazy. Especially now during the COVID-19 outbreak, which transforms business in an extremely fast way. I have heard that recently some big CEO has told that staying away from the cloud today would be like planing to extinct soon. Maybe this is too radical for me, but it gives a sense of mindset shift and technological revolution we are approaching today. I am not sure how it looks like at your place, but in my location, in Poland, after full lockdown, all educational activities have been moved from analog to digital. The teachers must deliver e-schooling via Zoom or on the other platforms. On the other hand, the shops are adapting very quickly to provide a nice and easy e-commerce experience with the delivery of all goodies at your footsteps in your house. For example, I was shocked it wasn’t possible to buy new shoes in the mall for my kids (full lockdown means malls are closed). The spring has approached and their feet were just bigger. There were no excuses. I have done it via e-shop…

And here is a big question. Who will build this brand new world of e-shops, e-schools, e-something, etc? There will be a massive need for cloud developers. The army of flexible programmers capable to utilize all of the features of the modern clouds. That is is why it is worth spending time on this topic – learn about OCI Event Services and OCI Functions,  explore OCI API Gateway, Oracle Kubernetes Engine (OKE),  OCI Autonomous Databases and  OCI Web Application Firewall (WAF). And of course, it is a good idea to get skills and hands-on experience with Terraform (here is my tutorial).

What do you think about it? I am very excited about your feedback in this field…

One more thing… Please remember to subscribe FoggyKitchen YT channel. 🙂

Martin, The Cook.

USE OCI OBJECT STORAGE TO STORE TERRAFORM STATE FILES

download.png

Introduction

Terraform allows you to deploy infrastructure very quickly, and you can find thousands of script/modules/snippets on the web to get you started in a snap.

Once you move from the “Let’s try this at home” phase to a more “Let’s do this for a real environment” oriented phase you will have to make a lot of decisions about how you are going to maintain your infrastructure and how you are going to manage change, possibly in a non destructive way.

Nothing like dropping a production environment on a Friday evening while you are testing a small change in your dev environment will make you realise how badly you need this when moving past the Terraform learning phase.

One important decision during this transition to real world use of terraform is the location where you are going to store a Terraform project state file.

In this tutorial you will learn how to configure OCI Object storage as a remote backend for a sample terraform project, this will allow you to replicate the setup for your projects.

What is a terraform state file?

Terraform must store state information about your managed infrastructure and configuration. This state is used by Terraform to map real world resources to your configuration, keep track of metadata, and to improve performance for large infrastructures.

This state is stored by default in a local file named “terraform.tfstate”, but it can also be stored remotely, which is essential when more than one person is working on the same project. With remote state, Terraform writes the state data to a remote data store, which can then be shared between all members of a team, encrypted, versioned and secured. Terraform supports storing state in Terraform CloudHashiCorp Consul, Amazon S3, Alibaba Cloud OSS, and other providers.

If you want to dig deeper into why Terraform needs a state file to work you can have a look here: https://www.terraform.io/docs/state/purpose.html

Terraform remote state file and OCI support

OCI object storage is not supported natively for storing state files, but since it has an AWS S3 compatibility layer we can use it all the same

Requirements

An Active Oracle Cloud Tenancy

If you do not have one, you can enroll here for a trial subscription that includes $300 of free credits

An Oracle Cloud Shell Instance

The OCI Cloud Shell is a web browser-based terminal accessible from the Oracle Cloud Console. Cloud Shell is free to use (within monthly tenancy limits), and provides access to a Linux shell, with a pre-authenticated Oracle Cloud Infrastructure CLI and other useful tools for following Oracle Cloud Infrastructure service tutorials and labs. Cloud Shell is a feature available to all OCI users, accessible from the Console. Your Cloud Shell will appear in the Oracle Cloud Console as a persistent frame of the Console and will stay active as you navigate to different pages of the Console. You can find detailed instructions on how to get started with the OCI Cloud Shell here

cloudshell-1.png

Create an initial working dir:

mkdir fk-terraform-objectstorage
cd fk-terraform-objectstorage/

A test terraform project

git clone https://github.com/mattiarossi/terraform-oci-default-vcn/


Cloning into 'terraform-oci-default-vcn'...
remote: Enumerating objects: 61, done.
remote: Counting objects: 100% (61/61), done.
remote: Compressing objects: 100% (36/36), done.
remote: Total 75 (delta 27), reused 44 (delta 14), pack-reused 14
Unpacking objects: 100% (75/75), done.


cd terraform-oci-default-vcn/examples/vcn_default/

The test project will deploy a simple VCN in a compartment of your choice, configurable in the project variables, and will store the state file in an OCI Object storage Bucket that needs to be created in advance

An OCI Object Storage Bucket

The bucket needs to be created in the Compartment that is designated on OCI to be the target for the AWS S3 compatibility layer. You can check the active S3 compartment by using the following command:

oci os ns  get-metadata --output=table
+-------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------+-----------+
| default-s3-compartment-id                                                           | default-swift-compartment-id                                                        | namespace |
+-------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------+-----------+
| ocid1.compartment.oc1..aaaaaaa...4nxhlvbbbbbbb3kpmffof6vba                          | ocid1.compartment.oc1..aaaaaaa...hlvbbbbbbbbpmffof6vba                              | mytenancy |
+-------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------+-----------+

We will be using a bucket named tf-fk-test-01

This command will create a bucket named tf-fk-test-01 in the AWS s3 compatibility layer compartment:

oci os bucket create --compartment-id ocid1.compartment.oc1..aaaaaaa...4nxhlvb3luoh3kpmffof6vba --name tf-fk-test-01

A properly configured AWS S3 compatibility environment

In order to be able to access the Object Storage bucket using the S3 compatibility layer, you need to generate a set of access credentials, and the oci cli command needed for creating these needs to know the current user ocid:

export ME=mattia.rossi@myemail.com (my tenancy username)
oci iam user list --query "data[?\"name\"=='$ME']".{"name:\"name\",id:\"id\""} --output table

+------------------------------------------------------------------------------+--------------------------+
| id                                                                           | name                     |
+------------------------------------------------------------------------------+--------------------------+
| ocid1.user.oc1..aaaaaaa..................................tojgy2qybdkcayvnxsq | mattia.rossi@myemail.com |
+------------------------------------------------------------------------------+--------------------------+

export OCIUID=ocid1.user.oc1..aaaaaaa..................................tojgy2qybdkcayvnxsq

The following command will reuse the ocid variable to create the credentials needed to setup the AWS compatibility layer (please note that these are examples only and you will need to insert the proper secrets generated by the oci cli in the export command

oci iam customer-secret-key create --user-id $OCIUID --display-name 'key-tf-test' --query "data".{"AWS_ACCESS_KEY_ID:\"id\",AWS_SECRET_ACCESS_KEY:\"key\""} --output=table

+------------------------------------------+----------------------------------------------+
| AWS_ACCESS_KEY_ID                        | AWS_SECRET_ACCESS_KEY                        |
+------------------------------------------+----------------------------------------------+
| fd9bcbb0.........................93e7139 | kS/...............................PRDoGX1NY= |
+------------------------------------------+----------------------------------------------+

export AWS_ACCESS_KEY_ID=fd9bcbb0.........................93e7139
export AWS_SECRET_ACCESS_KEY=kS/...............................PRDoGX1NY=

Environment variables

The only other environment variable needed by the project for authenticating is the tenancy ocid, that can be set up using this command:

export TF_VAR_tenancy_ocid=`cat /etc/oci/config  | grep tenancy | uniq | cut -d '=' -f 2`

Terraform project config

Prepare one variable file named “terraform.tfvars” with the configuration information. The content of “terraform.tfvars” should look something like the following:

$ cat terraform.tfvars
# Region
region = "eu-frankfurt-1"

# Compartment
compartment_ocid = ""

# VCN Configurations
vcn_display_name = "testVCN"
vcn_cidr = "10.0.0.0/16"

Use the following script to retrieve the ocid of the compartment where you want to deploy the test VCN:

export COMPARTMENT=my-compartment
oci iam compartment list --query "data[?\"name\"=='$COMPARTMENT']".{"name:\"name\",id:\"id\""} --output=table
+-------------------------------------------------------------------------------------+-----------------------+
| id                                                                                  | name                  |
+-------------------------------------------------------------------------------------+-----------------------+
| ocid1.compartment.oc1..aaaaaaaabmc54lgslm..........................3hygseg6qeh5pvwq | my-compartment        |
+-------------------------------------------------------------------------------------+-----------------------+

Edit the file vcn_default.tf, and update the terraform backend section to match your OCI setup:

terraform {
  backend "s3" {
    endpoint                    = "https://__mytenancy__.compat.objectstorage.__region__.oraclecloud.com"
    skip_metadata_api_check     = true
    skip_region_validation      = true
    force_path_style            = true
    skip_credentials_validation = true
    bucket                      = "__mybucket__"
    key                         = "__key__"
    region                      = "__region__"
  }
}

where:

  • __mytenancy__ is your tenancy name
  • __region__ is the region where you created the Object Storage bucket
  • __mybucket__ is the name of the bucket you created
  • key is the name of the bucket entry that will hold your terraform state file (for example: terraform/state/oci/vcn/testVCN/terraform.tfstate)

Then apply the example using the following commands:

terraform init
terraform plan
terraform apply

Init:

mattia_ros@cloudshell:vcn_default (eu-frankfurt-1)$ terraform init
Initializing modules...
- vcn in ../..

Initializing the backend...

Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "oci" (hashicorp/oci) 3.74.0...

The following providers do not have any version constraints in configuration,
so the latest version was installed.

To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.

* provider.oci: version = "~> 3.74"

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Plan:

mattia_ros@cloudshell:vcn_default (eu-frankfurt-1)$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

module.vcn.data.oci_identity_availability_domains.this: Refreshing state...

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create

Terraform will perform the following actions:

# module.vcn.oci_core_default_route_table.this will be created
+ resource "oci_core_default_route_table" "this" {
+ defined_tags = (known after apply)
+ display_name = (known after apply)
+ freeform_tags = (known after apply)
+ id = (known after apply)
+ manage_default_resource_id = (known after apply)
+ state = (known after apply)
+ time_created = (known after apply)

+ route_rules {
+ cidr_block = (known after apply)
+ description = (known after apply)
+ destination = "0.0.0.0/0"
+ destination_type = (known after apply)
+ network_entity_id = (known after apply)
}
}

# module.vcn.oci_core_internet_gateway.this will be created
+ resource "oci_core_internet_gateway" "this" {
+ compartment_id = "ocid1.compartment.oc1..aaaaaaaan2lhk5eqy2hvum45ahjsvgs2sz7x72ur5kw4euwe6my2226mk5ia"
+ defined_tags = (known after apply)
+ display_name = (known after apply)
+ enabled = true
+ freeform_tags = (known after apply)
+ id = (known after apply)
+ state = (known after apply)
+ time_created = (known after apply)
+ vcn_id = (known after apply)
}

# module.vcn.oci_core_subnet.this[0] will be created
+ resource "oci_core_subnet" "this" {
+ availability_domain = "okVf:EU-FRANKFURT-1-AD-1"
+ cidr_block = "192.168.0.0/28"
+ compartment_id = "ocid1.compartment.oc1..aaaaaaaan2lhk5eqy2hvum45ahjsvgs2sz7x72ur5kw4euwe6my2226mk5ia"
+ defined_tags = (known after apply)
+ dhcp_options_id = (known after apply)
+ display_name = "Default Subnet okVf:EU-FRANKFURT-1-AD-1"
+ dns_label = "subnet1"
+ freeform_tags = (known after apply)
+ id = (known after apply)
+ ipv6cidr_block = (known after apply)
+ ipv6public_cidr_block = (known after apply)
+ ipv6virtual_router_ip = (known after apply)
+ prohibit_public_ip_on_vnic = (known after apply)
+ route_table_id = (known after apply)
+ security_list_ids = (known after apply)
+ state = (known after apply)
+ subnet_domain_name = (known after apply)
+ time_created = (known after apply)
+ vcn_id = (known after apply)
+ virtual_router_ip = (known after apply)
+ virtual_router_mac = (known after apply)
}

# module.vcn.oci_core_subnet.this[1] will be created
+ resource "oci_core_subnet" "this" {
+ availability_domain = "okVf:EU-FRANKFURT-1-AD-2"
+ cidr_block = "192.168.0.16/28"
+ compartment_id = "ocid1.compartment.oc1..aaaaaaaan2lhk5eqy2hvum45ahjsvgs2sz7x72ur5kw4euwe6my2226mk5ia"
+ defined_tags = (known after apply)
+ dhcp_options_id = (known after apply)
+ display_name = "Default Subnet okVf:EU-FRANKFURT-1-AD-2"
+ dns_label = "subnet2"
+ freeform_tags = (known after apply)
+ id = (known after apply)
+ ipv6cidr_block = (known after apply)
+ ipv6public_cidr_block = (known after apply)
+ ipv6virtual_router_ip = (known after apply)
+ prohibit_public_ip_on_vnic = (known after apply)
+ route_table_id = (known after apply)
+ security_list_ids = (known after apply)
+ state = (known after apply)
+ subnet_domain_name = (known after apply)
+ time_created = (known after apply)
+ vcn_id = (known after apply)
+ virtual_router_ip = (known after apply)
+ virtual_router_mac = (known after apply)
}

# module.vcn.oci_core_subnet.this[2] will be created
+ resource "oci_core_subnet" "this" {
+ availability_domain = "okVf:EU-FRANKFURT-1-AD-3"
+ cidr_block = "192.168.0.32/28"
+ compartment_id = "ocid1.compartment.oc1..aaaaaaaan2lhk5eqy2hvum45ahjsvgs2sz7x72ur5kw4euwe6my2226mk5ia"
+ defined_tags = (known after apply)
+ dhcp_options_id = (known after apply)
+ display_name = "Default Subnet okVf:EU-FRANKFURT-1-AD-3"
+ dns_label = "subnet3"
+ freeform_tags = (known after apply)
+ id = (known after apply)
+ ipv6cidr_block = (known after apply)
+ ipv6public_cidr_block = (known after apply)
+ ipv6virtual_router_ip = (known after apply)
+ prohibit_public_ip_on_vnic = (known after apply)
+ route_table_id = (known after apply)
+ security_list_ids = (known after apply)
+ state = (known after apply)
+ subnet_domain_name = (known after apply)
+ time_created = (known after apply)
+ vcn_id = (known after apply)
+ virtual_router_ip = (known after apply)
+ virtual_router_mac = (known after apply)
}

# module.vcn.oci_core_vcn.this will be created
+ resource "oci_core_vcn" "this" {
+ cidr_block = "192.168.0.0/25"
+ compartment_id = "ocid1.compartment.oc1..aaaaaaaan2lhk5eqy2hvum45ahjsvgs2sz7x72ur5kw4euwe6my2226mk5ia"
+ default_dhcp_options_id = (known after apply)
+ default_route_table_id = (known after apply)
+ default_security_list_id = (known after apply)
+ defined_tags = (known after apply)
+ display_name = "TEST-FK-VCN"
+ dns_label = "vcn"
+ freeform_tags = (known after apply)
+ id = (known after apply)
+ ipv6cidr_block = (known after apply)
+ ipv6public_cidr_block = (known after apply)
+ is_ipv6enabled = (known after apply)
+ state = (known after apply)
+ time_created = (known after apply)
+ vcn_domain_name = (known after apply)
}

Plan: 6 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

Apply:

mattia_ros@cloudshell:vcn_default (eu-frankfurt-1)$ terraform apply
module.vcn.data.oci_identity_availability_domains.this: Refreshing state...

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create

Terraform will perform the following actions:

# module.vcn.oci_core_default_route_table.this will be created
+ resource "oci_core_default_route_table" "this" {
+ defined_tags = (known after apply)
+ display_name = (known after apply)
+ freeform_tags = (known after apply)
+ id = (known after apply)
+ manage_default_resource_id = (known after apply)
+ state = (known after apply)
+ time_created = (known after apply)

+ route_rules {
+ cidr_block = (known after apply)
+ description = (known after apply)
+ destination = "0.0.0.0/0"
+ destination_type = (known after apply)
+ network_entity_id = (known after apply)
}
}

# module.vcn.oci_core_internet_gateway.this will be created
+ resource "oci_core_internet_gateway" "this" {
+ compartment_id = "ocid1.compartment.oc1..aaaaaaaan2lhk5eqy2hvum45ahjsvgs2sz7x72ur5kw4euwe6my2226mk5ia"
+ defined_tags = (known after apply)
+ display_name = (known after apply)
+ enabled = true
+ freeform_tags = (known after apply)
+ id = (known after apply)
+ state = (known after apply)
+ time_created = (known after apply)
+ vcn_id = (known after apply)
}

# module.vcn.oci_core_subnet.this[0] will be created
+ resource "oci_core_subnet" "this" {
+ availability_domain = "okVf:EU-FRANKFURT-1-AD-1"
+ cidr_block = "192.168.0.0/28"
+ compartment_id = "ocid1.compartment.oc1..aaaaaaaan2lhk5eqy2hvum45ahjsvgs2sz7x72ur5kw4euwe6my2226mk5ia"
+ defined_tags = (known after apply)
+ dhcp_options_id = (known after apply)
+ display_name = "Default Subnet okVf:EU-FRANKFURT-1-AD-1"
+ dns_label = "subnet1"
+ freeform_tags = (known after apply)
+ id = (known after apply)
+ ipv6cidr_block = (known after apply)
+ ipv6public_cidr_block = (known after apply)
+ ipv6virtual_router_ip = (known after apply)
+ prohibit_public_ip_on_vnic = (known after apply)
+ route_table_id = (known after apply)
+ security_list_ids = (known after apply)
+ state = (known after apply)
+ subnet_domain_name = (known after apply)
+ time_created = (known after apply)
+ vcn_id = (known after apply)
+ virtual_router_ip = (known after apply)
+ virtual_router_mac = (known after apply)
}

# module.vcn.oci_core_subnet.this[1] will be created
+ resource "oci_core_subnet" "this" {
+ availability_domain = "okVf:EU-FRANKFURT-1-AD-2"
+ cidr_block = "192.168.0.16/28"
+ compartment_id = "ocid1.compartment.oc1..aaaaaaaan2lhk5eqy2hvum45ahjsvgs2sz7x72ur5kw4euwe6my2226mk5ia"
+ defined_tags = (known after apply)
+ dhcp_options_id = (known after apply)
+ display_name = "Default Subnet okVf:EU-FRANKFURT-1-AD-2"
+ dns_label = "subnet2"
+ freeform_tags = (known after apply)
+ id = (known after apply)
+ ipv6cidr_block = (known after apply)
+ ipv6public_cidr_block = (known after apply)
+ ipv6virtual_router_ip = (known after apply)
+ prohibit_public_ip_on_vnic = (known after apply)
+ route_table_id = (known after apply)
+ security_list_ids = (known after apply)
+ state = (known after apply)
+ subnet_domain_name = (known after apply)
+ time_created = (known after apply)
+ vcn_id = (known after apply)
+ virtual_router_ip = (known after apply)
+ virtual_router_mac = (known after apply)
}

# module.vcn.oci_core_subnet.this[2] will be created
+ resource "oci_core_subnet" "this" {
+ availability_domain = "okVf:EU-FRANKFURT-1-AD-3"
+ cidr_block = "192.168.0.32/28"
+ compartment_id = "ocid1.compartment.oc1..aaaaaaaan2lhk5eqy2hvum45ahjsvgs2sz7x72ur5kw4euwe6my2226mk5ia"
+ defined_tags = (known after apply)
+ dhcp_options_id = (known after apply)
+ display_name = "Default Subnet okVf:EU-FRANKFURT-1-AD-3"
+ dns_label = "subnet3"
+ freeform_tags = (known after apply)
+ id = (known after apply)
+ ipv6cidr_block = (known after apply)
+ ipv6public_cidr_block = (known after apply)
+ ipv6virtual_router_ip = (known after apply)
+ prohibit_public_ip_on_vnic = (known after apply)
+ route_table_id = (known after apply)
+ security_list_ids = (known after apply)
+ state = (known after apply)
+ subnet_domain_name = (known after apply)
+ time_created = (known after apply)
+ vcn_id = (known after apply)
+ virtual_router_ip = (known after apply)
+ virtual_router_mac = (known after apply)
}

# module.vcn.oci_core_vcn.this will be created
+ resource "oci_core_vcn" "this" {
+ cidr_block = "192.168.0.0/25"
+ compartment_id = "ocid1.compartment.oc1..aaaaaaaan2lhk5eqy2hvum45ahjsvgs2sz7x72ur5kw4euwe6my2226mk5ia"
+ default_dhcp_options_id = (known after apply)
+ default_route_table_id = (known after apply)
+ default_security_list_id = (known after apply)
+ defined_tags = (known after apply)
+ display_name = "TEST-FK-VCN"
+ dns_label = "vcn"
+ freeform_tags = (known after apply)
+ id = (known after apply)
+ ipv6cidr_block = (known after apply)
+ ipv6public_cidr_block = (known after apply)
+ is_ipv6enabled = (known after apply)
+ state = (known after apply)
+ time_created = (known after apply)
+ vcn_domain_name = (known after apply)
}

Plan: 6 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.

Enter a value: yes

module.vcn.oci_core_vcn.this: Creating...
module.vcn.oci_core_vcn.this: Creation complete after 0s [id=ocid1.vcn.oc1.eu-frankfurt-1.amaaaaaavyxeuaqaavuidjkp2un5v7fgodw63pytlkaxvrfs5coucgfqmgfa]
module.vcn.oci_core_internet_gateway.this: Creating...
module.vcn.oci_core_subnet.this[2]: Creating...
module.vcn.oci_core_subnet.this[1]: Creating...
module.vcn.oci_core_subnet.this[0]: Creating...
module.vcn.oci_core_subnet.this[0]: Creation complete after 0s [id=ocid1.subnet.oc1.eu-frankfurt-1.aaaaaaaa2feksp66yxjrj2twxypjbuk5blefju2ghkk34z4hvex57obkxzqa]
module.vcn.oci_core_subnet.this[2]: Creation complete after 1s [id=ocid1.subnet.oc1.eu-frankfurt-1.aaaaaaaadqbslze7krqigudqg6ajhxhxddpmhvaottievimyk6jdxz754pta]
module.vcn.oci_core_subnet.this[1]: Creation complete after 1s [id=ocid1.subnet.oc1.eu-frankfurt-1.aaaaaaaaoqrhcrv743sdt6norse7wql7rmin3ca6amoigqonvsejuoavnjtq]
module.vcn.oci_core_internet_gateway.this: Creation complete after 2s [id=ocid1.internetgateway.oc1.eu-frankfurt-1.aaaaaaaau4sdjijm4b3miu2hwf5lcbkyb7eaea62qg7dpaar2udn4u4zhfnq]
module.vcn.oci_core_default_route_table.this: Creating...
module.vcn.oci_core_default_route_table.this: Creation complete after 0s [id=ocid1.routetable.oc1.eu-frankfurt-1.aaaaaaaa5ag3j4kbkb763c3qorcowy5r3swyileytlyhagkv7mqragxysgma]

Apply complete! Resources: 6 added, 0 changed, 0 destroyed.

Outputs:

default_dhcp_options_id = ocid1.dhcpoptions.oc1.eu-frankfurt-1.aaaaaaaabzs2tgdpnq6jrd2p67lks44qovk272uxuoi3kgciib62fyflgyja
default_route_table_id = ocid1.routetable.oc1.eu-frankfurt-1.aaaaaaaa5ag3j4kbkb763c3qorcowy5r3swyileytlyhagkv7mqragxysgma
default_security_list_id = ocid1.securitylist.oc1.eu-frankfurt-1.aaaaaaaadm7oi73fwjx2kiiqbc7ucsuvqcmxfjis27at4p3tyhuopod5tqoq
internet_gateway_id = ocid1.internetgateway.oc1.eu-frankfurt-1.aaaaaaaau4sdjijm4b3miu2hwf5lcbkyb7eaea62qg7dpaar2udn4u4zhfnq
subnet_ids = [
"ocid1.subnet.oc1.eu-frankfurt-1.aaaaaaaa2feksp66yxjrj2twxypjbuk5blefju2ghkk34z4hvex57obkxzqa",
"ocid1.subnet.oc1.eu-frankfurt-1.aaaaaaaaoqrhcrv743sdt6norse7wql7rmin3ca6amoigqonvsejuoavnjtq",
"ocid1.subnet.oc1.eu-frankfurt-1.aaaaaaaadqbslze7krqigudqg6ajhxhxddpmhvaottievimyk6jdxz754pta",
]
vcn_id = ocid1.vcn.oc1.eu-frankfurt-1.amaaaaaavyxeuaqaavuidjkp2un5v7fgodw63pytlkaxvrfs5coucgfqmgfa

Now let’s check the local terraform state file:

mattia_ros@cloudshell:vcn_default (eu-frankfurt-1)$ cat .terraform/terraform.tfstate 
{
"version": 3,
"serial": 1,
"lineage": "1250a1d2-3e20-124b-7c25-524f67566052",
"backend": {
"type": "s3",
"config": {
"access_key": null,
"acl": null,
"assume_role_policy": null,
"bucket": "tf-fk-test-01",
"dynamodb_endpoint": null,
"dynamodb_table": null,
"encrypt": null,
"endpoint": "https://mytenancy.compat.objectstorage.eu-frankfurt-1.oraclecloud.com",
"external_id": null,
"force_path_style": true,
"iam_endpoint": null,
"key": "terraform/state/oci/vcn/testVCN/terraform.tfstate",
"kms_key_id": null,
"lock_table": null,
"max_retries": null,
"profile": null,
"region": "eu-frankfurt-1",
"role_arn": null,
"secret_key": null,
"session_name": null,
"shared_credentials_file": null,
"skip_credentials_validation": true,
"skip_get_ec2_platforms": null,
"skip_metadata_api_check": true,
"skip_region_validation": true,
"skip_requesting_account_id": null,
"sse_customer_key": null,
"sts_endpoint": null,
"token": null,
"workspace_key_prefix": null
},
"hash": 1758364997
},
"modules": [
{
"path": [
"root"
],
"outputs": {},
"resources": {},
"depends_on": []
}
]
}

And let’s check the Object storage bucket:

oci os object list -bn tf-fk-test-01 --output=table
+------+--------------------------+---------------------------------------------------+-------+----------------------------------+---------------+
| etag | md5 | name | size | time-created | time-modified |
+------+--------------------------+---------------------------------------------------+-------+----------------------------------+---------------+
| None | PRXsobIN......efUorHfg== | terraform/state/oci/vcn/testVCN/terraform.tfstate | 14161 | 2020-05-08T18:43:19.428000+00:00 | None |
+------+--------------------------+---------------------------------------------------+-------+----------------------------------+---------------+
prefixes: []

.. and we’re done, you have configured terraform to store state files remotely on OCI.

If you want to use the same pattern outside the OCI cloud shell you will need to configure the oci provider to use the usual API key/fingerprint combo

Have Fun!

Mattia

OCI-Ballerina module

What is Ballerina?

Ballerina is an open-source, general-purpose, cloud-native programming language that incorporates concepts of distributed systems and micro-service development with both textual and graphical syntax. So why a completely new language? In the current micro-services world, I believe the language should have more capabilities to deal with creating containers, storing and deploying them to cloud platforms. The trend seems to be similar to the frameworks around JavaScript. Angular and React were born to incorporate the best practices to write clean and efficient code. One could argue that those are frameworks or libraries that could also be added to the language, but what if those features are available already in the language? It is all up to the developer to decide to choose the capabilities available in the language (Remember “import” in java?). For more details and discussions, check here

The OCI-Ballerina GitHub module:

The GitHub module provides the language to interact with OCI resources for a seamless native cloud application development process. The module would enable a user to efficiently provision and manage infrastructure resources on OCI. For example, a user might write business logic code or configuration code (e.g. provisioning scripts).

Risk Events Service from Oracle CASB

The Oracle Cloud Access Security Broker gives visibility into the entire cloud stack and acts as the security automation tool for the IT needs. For more details, check the details here. There are different modules like threat detection, predictivity analytics, and Security configuration available in CASB. In this example, we will take advantage of CASB’s risk events API to get the risk event data for the cloud tenancy. We will save this data to OCI Object Storage, which then could be leveraged to integrate with any other security and monitoring systems like Splunk, QRadar, or Oracle Management Cloud.

Prerequisites:

Once the module is downloaded from the GitHub, we need to configure the tenancy details as mentioned below. Once configured and tested, we can create an ociClient object to interact with OCI.

Image – Prerequisites to interact with OCI

JSON to CSV Transform:

Once we acquire the risk event data in JSON format, we can take the advantage of the native JSON to CSV file (shown below) converter available in the Ballerina language, to upload the file to object storage.

Image: JSON to CSV converter built in the language

Image: CSV file uploaded to OCI object storage

Summary:

To create a micro-service that fetches risk event data from Oracle CASB, we took advantage of the GitHub module for OCI-Ballerina interaction. We obtained the risk events data in a JSON format, converted into a CSV file, and uploaded it to the OCI object storage. The GitHub module also allows a user to interact with other OCI resources to develop and deploy on OCI.

How to integrate OCI Event Service and OCI Functions with Terraform

Yesterday night I have passed  Oracle Cloud Infrastructure Developer 2020 Certified Associate exam. For this exam, besides Terraform, I had to learn about Oracle Kubernetes Engine (OKE) and OCI Functions. Recently I have had the opportunity to explore OCI Event Service. The outcome of this experiment I have encoded in a brand new dish (GitHub repo here). The idea is simple. Successful execution of the manual backup of compute instance’s block volume or boot volume leads to automatically populated tagging information, replicated to backup resource from the corresponding volume resource. Worth to add that for automated volumes backup, recently this feature has been implemented natively in OCI, but for manual backups, we are still waiting for this feature. My best guess here is we can treat this experiment for now as a foundation for further experiments in your kitchen. So please, treat is as a toy, just for fun 🙂

Bon Appetit,

Martin, The Cook.