Create a custom OCI image using Packer and the OCI Cloud Shell
Today we would like to show you how to bake a custom image in OCI using packer, a tool that complements terraform in creating Infrastructure as a code deployment.
We will use an OCI standard Oracle Autonomous Linux Image, and create a custom one that consists of the original image, plus the Oracle instant client libraries and tools like sqlplus and impdp/expdp. At the end of the process, a new custom image will be available for you to deploy in your OCI tenancy.
What is Packer?
Required components:
An Active Oracle Cloud Tenancy
An Oracle Cloud Shell Instance
mrossi@cloudshell:~ (us-phoenix-1)$ mkdir packer-cloud-test
mrossi@cloudshell:~ (us-phoenix-1)$ cd packer-cloud-test/
An Existing public subnet in an existing Virtual Cloud network
For an example of how to do that using terraform, you can use this tutorial: OCI cli Practice 4: Create another VCN with one public subnet
Packer
mrossi@cloudshell:packer-cloud-test (us-phoenix-1)$ wget "https://releases.hashicorp.com/packer/1.5.4/packer_1.5.4_linux_amd64.zip"
--2020-03-05 12:59:15-- https://releases.hashicorp.com/packer/1.5.4/packer_1.5.4_linux_amd64.zip
Resolving releases.hashicorp.com (releases.hashicorp.com)... 151.101.25.183, 2a04:4e42:6::439
Connecting to releases.hashicorp.com (releases.hashicorp.com)|151.101.25.183|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 48051150 (46M) [application/zip]
Saving to: ‘packer_1.5.4_linux_amd64.zip’
100%[===============================================================================================================================================>] 48,051,150 53.2MB/s in 0.9s
2020-03-05 12:59:16 (53.2 MB/s) - ‘packer_1.5.4_linux_amd64.zip’ saved [48051150/48051150]
mrossi@cloudshell:packer-cloud-test (us-phoenix-1)$ unzip packer_1.5.4_linux_amd64.zip
Archive: packer_1.5.4_linux_amd64.zip
inflating: packer
mrossi@cloudshell:packer-cloud-test (us-phoenix-1)$ ./packer
Usage: packer [--version] [--help] <command> [<args>]
Available commands are:
build build image(s) from template
console creates a console for testing variable interpolation
fix fixes templates from old versions of packer
inspect see components of a template
validate check that a template is valid
version Prints the Packer version
mrossi@cloudshell:packer-cloud-test (us-phoenix-1)$
Packer build file
mrossi@cloudshell:oci-packer-customimage (us-phoenix-1)$ wget https://gist.githubusercontent.com/mattiarossi/8321dc84e305b59a7969cf7bf856909f/raw/33ed77d20a42cc41b75a63178ec0cf0e48992a6e/packer-oci-autonomous-instantclient.json
--2020-03-05 12:04:15-- https://gist.githubusercontent.com/mattiarossi/8321dc84e305b59a7969cf7bf856909f/raw/33ed77d20a42cc41b75a63178ec0cf0e48992a6e/packer-oci-autonomous-instantclient.json
Resolving gist.githubusercontent.com (gist.githubusercontent.com)... 151.101.196.133
Connecting to gist.githubusercontent.com (gist.githubusercontent.com)|151.101.196.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1698 (1.7K) [text/plain]
Saving to: ‘packer-oci-autonomous-instantclient.json’
100%[==============================================================================================================================================================================================================================>] 1,698 --.-K/s in 0s
2020-03-05 12:04:15 (7.32 MB/s) - ‘packer-oci-autonomous-instantclient.json�� saved [1698/1698]
mrossi@cloudshell:oci-packer-customimage (us-phoenix-1)$ cat packer-oci-autonomous-instantclient.json
{
"variables": {
"user_ocid":"{{env `user_ocid`}}",
"tenancy_ocid": "{{env `tenancy_ocid`}}",
"fingerprint":"{{env `fingerprint`}}",
"private_key_path":"{{env `private_key_path`}}",
"availability_domain": "{{env `availability_domain`}}",
"region": "{{env `region`}}",
"base_image_ocid": "{{env `base_image_ocid`}}",
"compartment_ocid": "{{env `compartment_ocid`}}",
"subnet_ocid": "{{env `subnet_ocid`}}"
},
"builders": [
{
"user_ocid":"{{user `user_ocid`}}",
"tenancy_ocid": "{{user `tenancy_ocid`}}",
"fingerprint":"{{user `fingerprint`}}",
"key_file":"{{user `private_key_path`}}",
"availability_domain": "{{user `availability_domain`}}",
"region": "{{user `region`}}",
"base_image_ocid": "{{user `base_image_ocid`}}",
"compartment_ocid": "{{user `compartment_ocid`}}",
"image_name": "autonomous-instantclient-demo",
"shape": "VM.Standard2.1",
"ssh_username": "opc",
"ssh_password": "aaaaatrbfgyreyuuyreb",
"subnet_ocid": "{{user `subnet_ocid`}}",
"type": "oracle-oci"
}
],
"provisioners": [
{
"type": "shell",
"inline": [
"sudo -u root yum-config-manager --add-repo http://yum.oracle.com/repo/OracleLinux/OL7/oracle/instantclient/x86_64",
"sudo -u root yum -y erase oracle-instantclient18.3-basic",
"sudo -u root yum -y install oracle-instantclient19.5-devel oracle-instantclient19.5-tools oracle-instantclient19.5-jdbc oracle-instantclient19.5-sqlplus",
"sqlplus -v",
"/usr/lib/oracle/19.5/client64/bin/imp HELP=Y"
],
"pause_before": "5s"
}
]
}
Setup Environment Variables needed by Packer:
You will have to follow these instructions to create valid API keys:
Once you have successfully created valid API keys, you will have the following details handy:
- user_ocid : ID of the user that has been configured with valid API keys
- tenancy_ocid: ID of the tenancy where the Custom image needs to be created, if it is the same as the Cloud console tenancy use the provided script to retrieve it directly
- fingerprint: your API key Fingerprint
- private_key_path: a copy of the private key that has been configured for API access
In addition to that, some other env variables will need to be set up to point the Packer builder to the correct OCI environment
- compartment_ocid : ID of the Compartment that will host the temporary Instance
- base_image_ocid : ID of the base image that will be used as a source, in our case: Autonomous Linux 7
- subnet_ocid : ID of the Subnet that will host
You can use this shell file as an example
mrossi@cloudshell:oci-packer-customimage (us-phoenix-1)$ cat setup_oci_vars.sh
export user_ocid="ocid1.user.oc1..<your user ocid>"
export tenancy_ocid="`cat /etc/oci/config | grep tenancy | uniq | cut -d "=" -f 2`"
export compartment_ocid="ocid1.compartment.oc1..<id of the compartment that will host the temporary instance>"
export fingerprint="<your API key fingerprint>"
export private_key_path="<path to your OCI api private key>"
export region="us-phoenix-1"
#Use `oci iam availability-domain list` to get a list ov the availability domains in the current region
export availability_domain="zTzD:PHX-AD-1"
#OCID of the base image
#Use:
#oci compute image list --compartment-id $compartment_ocid | jq '.data[] | {image:."display-name", id:."id"}| select ( .image | contains("Autonomous"))'
# to get a list of the available ocids in the current region, if you need to deploy an image in a different region, adjust the oci cli command or get the id from the Web console
export base_image_ocid="ocid1.image.oc1.phx.aaaaaaaa3ocrbp42shmxitpk5nqphpptnnvtk2grrkzfnap4idejfcpv2q4a"
#Subnet id of a suitable OCI Subnet, the packer client needs to be able to SSH to it in order to deploy additional configuration
#Use:
#oci network subnet list --compartment-id $compartment_ocid --vcn-id $vcn_id | jq '.data[] | {subnet:."display-name", id:."id"}'
#to get a list of subnets in a given VCN
export subnet_ocid="ocid1.subnet.oc1.phx....v7q7ybja"
And, once all fields have been updated, you need to source it:
mrossi@cloudshell:packer-cloud-test (us-phoenix-1)$ . setup_oci_vars.sh
mrossi@cloudshell:packer-cloud-test (us-phoenix-1)$ env | grep -e ocid -e key
compartment_ocid=ocid1.compartment.oc1..aaaa...f6vba
private_key_path=/tmp/oci_api_key.pem
base_image_ocid=ocid1.image.oc1.phx.aaa...jfcpv2q4a
user_ocid=ocid1.user.oc1..aaaaaaaavpt2mna...wfucqlqutg2na
tenancy_ocid=ocid1.tenancy.oc1..aaaa...3u3shj3kxq
subnet_ocid=ocid1.subnet.oc1.phx.aaaaa...bha44siv7q7ybja
Run packer
mrossi@cloudshell:packer-cloud-test (us-phoenix-1)$ ./packer build packer-oci-autonomous-instantclient.json
oracle-oci: output will be in this color.
==> oracle-oci: Creating temporary ssh key for instance...
==> oracle-oci: Creating instance...
==> oracle-oci: Created instance (ocid1.instance.oc1.phx.anyhqljtu24ak7acd3vzqmhazefic7zo6bebsbqz3xvmnq7e23vqexiex2hq).
==> oracle-oci: Waiting for instance to enter 'RUNNING' state...
==> oracle-oci: Instance 'RUNNING'.
==> oracle-oci: Instance has IP: 158.101.23.210.
==> oracle-oci: Using ssh communicator to connect: 158.101.23.210
==> oracle-oci: Waiting for SSH to become available...
==> oracle-oci: Connected to SSH!
==> oracle-oci: Pausing 5s before the next provisioner...
==> oracle-oci: Provisioning with shell script: /tmp/packer-shell958521454
oracle-oci: Loaded plugins: langpacks
oracle-oci: adding repo from: http://yum.oracle.com/repo/OracleLinux/OL7/oracle/instantclient/x86_64
oracle-oci:
oracle-oci: [yum.oracle.com_repo_OracleLinux_OL7_oracle_instantclient_x86_64]
oracle-oci: name=added from: http://yum.oracle.com/repo/OracleLinux/OL7/oracle/instantclient/x86_64
oracle-oci: baseurl=http://yum.oracle.com/repo/OracleLinux/OL7/oracle/instantclient/x86_64
oracle-oci: enabled=1
oracle-oci:
oracle-oci:
oracle-oci: Loaded plugins: langpacks, ulninfo
oracle-oci: Resolving Dependencies
oracle-oci: --> Running transaction check
oracle-oci: ---> Package oracle-instantclient18.3-basic.x86_64 0:18.3.0.0.0-2 will be erased
oracle-oci: --> Processing Dependency: oracle-instantclient18.3-basic >= 18.3.0.0.0 for package: python-cx_Oracle-7.0-1.0.1.el7.x86_64
oracle-oci: --> Running transaction check
oracle-oci: ---> Package python-cx_Oracle.x86_64 0:7.0-1.0.1.el7 will be erased
oracle-oci: --> Processing Dependency: python-cx_Oracle = 7.0 for package: python-oci-cli-2.9.0-1.el7.noarch
oracle-oci: --> Running transaction check
oracle-oci: ---> Package python-oci-cli.noarch 0:2.9.0-1.el7 will be erased
oracle-oci: --> Processing Dependency: python-oci-cli for package: al-config-1.0-3.el7.noarch
oracle-oci: --> Running transaction check
oracle-oci: ---> Package al-config.noarch 0:1.0-3.el7 will be erased
oracle-oci: --> Finished Dependency Resolution
oracle-oci:
oracle-oci: Dependencies Resolved
oracle-oci:
oracle-oci: ================================================================================
oracle-oci: Package Arch Version Repository
oracle-oci: Size
oracle-oci: ================================================================================
oracle-oci: Removing:
oracle-oci: oracle-instantclient18.3-basic x86_64 18.3.0.0.0-2 @al7 220 M
oracle-oci: Removing for dependencies:
oracle-oci: al-config noarch 1.0-3.el7 @al7 37 k
oracle-oci: python-cx_Oracle x86_64 7.0-1.0.1.el7 @al7 1.3 M
oracle-oci: python-oci-cli noarch 2.9.0-1.el7 @al7 9.9 M
oracle-oci:
oracle-oci: Transaction Summary
oracle-oci: ================================================================================
oracle-oci: Remove 1 Package (+3 Dependent packages)
oracle-oci:
oracle-oci: Installed size: 232 M
oracle-oci: Downloading packages:
oracle-oci: Running transaction check
oracle-oci: Running transaction test
oracle-oci: Transaction test succeeded
oracle-oci: Running transaction
oracle-oci: Erasing : al-config-1.0-3.el7.noarch 1/4
oracle-oci: Erasing : python-oci-cli-2.9.0-1.el7.noarch 2/4
oracle-oci: Erasing : python-cx_Oracle-7.0-1.0.1.el7.x86_64 3/4
oracle-oci: Erasing : oracle-instantclient18.3-basic-18.3.0.0.0-2.x86_64 4/4
oracle-oci: Verifying : python-oci-cli-2.9.0-1.el7.noarch 1/4
oracle-oci: Verifying : python-cx_Oracle-7.0-1.0.1.el7.x86_64 2/4
oracle-oci: Verifying : al-config-1.0-3.el7.noarch 3/4
oracle-oci: Verifying : oracle-instantclient18.3-basic-18.3.0.0.0-2.x86_64 4/4
oracle-oci:
oracle-oci: Removed:
oracle-oci: oracle-instantclient18.3-basic.x86_64 0:18.3.0.0.0-2
oracle-oci:
oracle-oci: Dependency Removed:
oracle-oci: al-config.noarch 0:1.0-3.el7 python-cx_Oracle.x86_64 0:7.0-1.0.1.el7
oracle-oci: python-oci-cli.noarch 0:2.9.0-1.el7
oracle-oci:
oracle-oci: Complete!
oracle-oci: Loaded plugins: langpacks, ulninfo
oracle-oci: Resolving Dependencies
oracle-oci: --> Running transaction check
oracle-oci: ---> Package oracle-instantclient19.5-devel.x86_64 0:19.5.0.0.0-1 will be installed
oracle-oci: --> Processing Dependency: oracle-instantclient19.5-basic >= 19.5.0.0.0 for package: oracle-instantclient19.5-devel-19.5.0.0.0-1.x86_64
oracle-oci: ---> Package oracle-instantclient19.5-jdbc.x86_64 0:19.5.0.0.0-1 will be installed
oracle-oci: ---> Package oracle-instantclient19.5-sqlplus.x86_64 0:19.5.0.0.0-1 will be installed
oracle-oci: ---> Package oracle-instantclient19.5-tools.x86_64 0:19.5.0.0.0-1 will be installed
oracle-oci: --> Running transaction check
oracle-oci: ---> Package oracle-instantclient19.5-basic.x86_64 0:19.5.0.0.0-1 will be installed
oracle-oci: --> Finished Dependency Resolution
oracle-oci:
oracle-oci: Dependencies Resolved
oracle-oci:
oracle-oci: ================================================================================
oracle-oci: Package Arch Version Repository Size
oracle-oci: ================================================================================
oracle-oci: Installing:
oracle-oci: oracle-instantclient19.5-devel x86_64 19.5.0.0.0-1 yum.oracle.com_repo_OracleLinux_OL7_oracle_instantclient_x86_64
oracle-oci: 598 k
oracle-oci: oracle-instantclient19.5-jdbc x86_64 19.5.0.0.0-1 yum.oracle.com_repo_OracleLinux_OL7_oracle_instantclient_x86_64
oracle-oci: 1.5 M
oracle-oci: oracle-instantclient19.5-sqlplus x86_64 19.5.0.0.0-1 yum.oracle.com_repo_OracleLinux_OL7_oracle_instantclient_x86_64
oracle-oci: 686 k
oracle-oci: oracle-instantclient19.5-tools x86_64 19.5.0.0.0-1 yum.oracle.com_repo_OracleLinux_OL7_oracle_instantclient_x86_64
oracle-oci: 818 k
oracle-oci: Installing for dependencies:
oracle-oci: oracle-instantclient19.5-basic x86_64 19.5.0.0.0-1 yum.oracle.com_repo_OracleLinux_OL7_oracle_instantclient_x86_64
oracle-oci: 51 M
oracle-oci:
oracle-oci: Transaction Summary
oracle-oci: ================================================================================
oracle-oci: Install 4 Packages (+1 Dependent package)
oracle-oci:
oracle-oci: Total download size: 55 M
oracle-oci: Installed size: 236 M
oracle-oci: Downloading packages:
oracle-oci: --------------------------------------------------------------------------------
oracle-oci: Total 44 MB/s | 55 MB 00:01
oracle-oci: Running transaction check
oracle-oci: Running transaction test
oracle-oci: Transaction test succeeded
oracle-oci: Running transaction
oracle-oci: Installing : oracle-instantclient19.5-basic-19.5.0.0.0-1.x86_64 1/5
oracle-oci: Installing : oracle-instantclient19.5-devel-19.5.0.0.0-1.x86_64 2/5
oracle-oci: Installing : oracle-instantclient19.5-jdbc-19.5.0.0.0-1.x86_64 3/5
oracle-oci: Installing : oracle-instantclient19.5-tools-19.5.0.0.0-1.x86_64 4/5
oracle-oci: Installing : oracle-instantclient19.5-sqlplus-19.5.0.0.0-1.x86_64 5/5
oracle-oci: Verifying : oracle-instantclient19.5-devel-19.5.0.0.0-1.x86_64 1/5
oracle-oci: Verifying : oracle-instantclient19.5-basic-19.5.0.0.0-1.x86_64 2/5
oracle-oci: Verifying : oracle-instantclient19.5-jdbc-19.5.0.0.0-1.x86_64 3/5
oracle-oci: Verifying : oracle-instantclient19.5-tools-19.5.0.0.0-1.x86_64 4/5
oracle-oci: Verifying : oracle-instantclient19.5-sqlplus-19.5.0.0.0-1.x86_64 5/5
oracle-oci:
oracle-oci: Installed:
oracle-oci: oracle-instantclient19.5-devel.x86_64 0:19.5.0.0.0-1
oracle-oci: oracle-instantclient19.5-jdbc.x86_64 0:19.5.0.0.0-1
oracle-oci: oracle-instantclient19.5-sqlplus.x86_64 0:19.5.0.0.0-1
oracle-oci: oracle-instantclient19.5-tools.x86_64 0:19.5.0.0.0-1
oracle-oci:
oracle-oci: Dependency Installed:
oracle-oci: oracle-instantclient19.5-basic.x86_64 0:19.5.0.0.0-1
oracle-oci:
oracle-oci: Complete!
oracle-oci:
oracle-oci: SQL*Plus: Release 19.0.0.0.0 - Production
oracle-oci: Version 19.5.0.0.0
oracle-oci:
==> oracle-oci:
==> oracle-oci: Import: Release 19.0.0.0.0 - Production on Thu Mar 5 11:02:37 2020
==> oracle-oci: Version 19.5.0.0.0
==> oracle-oci:
==> oracle-oci: Copyright (c) 1982, 2019, Oracle and/or its affiliates. All rights reserved.
==> oracle-oci:
==> oracle-oci:
==> oracle-oci:
==> oracle-oci: You can let Import prompt you for parameters by entering the IMP
==> oracle-oci: command followed by your username/password:
==> oracle-oci:
==> oracle-oci: Example: IMP SCOTT/TIGER
==> oracle-oci:
==> oracle-oci: Or, you can control how Import runs by entering the IMP command followed
==> oracle-oci: by various arguments. To specify parameters, you use keywords:
==> oracle-oci:
==> oracle-oci: Format: IMP KEYWORD=value or KEYWORD=(value1,value2,...,valueN)
==> oracle-oci: Example: IMP SCOTT/TIGER IGNORE=Y TABLES=(EMP,DEPT) FULL=N
==> oracle-oci: or TABLES=(T1:P1,T1:P2), if T1 is partitioned table
==> oracle-oci:
==> oracle-oci: USERID must be the first parameter on the command line.
==> oracle-oci:
==> oracle-oci: Keyword Description (Default) Keyword Description (Default)
==> oracle-oci: --------------------------------------------------------------------------
==> oracle-oci: USERID username/password FULL import entire file (N)
==> oracle-oci: BUFFER size of data buffer FROMUSER list of owner usernames
==> oracle-oci: FILE input files (EXPDAT.DMP) TOUSER list of usernames
==> oracle-oci: SHOW just list file contents (N) TABLES list of table names
==> oracle-oci: IGNORE ignore create errors (N) RECORDLENGTH length of IO record
==> oracle-oci: GRANTS import grants (Y) INCTYPE incremental import type
==> oracle-oci: INDEXES import indexes (Y) COMMIT commit array insert (N)
==> oracle-oci: ROWS import data rows (Y) PARFILE parameter filename
==> oracle-oci: LOG log file of screen output CONSTRAINTS import constraints (Y)
==> oracle-oci: DESTROY overwrite tablespace data file (N)
==> oracle-oci: INDEXFILE write table/index info to specified file
==> oracle-oci: SKIP_UNUSABLE_INDEXES skip maintenance of unusable indexes (N)
==> oracle-oci: FEEDBACK display progress every x rows(0)
==> oracle-oci: TOID_NOVALIDATE skip validation of specified type ids
==> oracle-oci: FILESIZE maximum size of each dump file
==> oracle-oci: STATISTICS import precomputed statistics (always)
==> oracle-oci: RESUMABLE suspend when a space related error is encountered(N)
==> oracle-oci: RESUMABLE_NAME text string used to identify resumable statement
==> oracle-oci: RESUMABLE_TIMEOUT wait time for RESUMABLE
==> oracle-oci: COMPILE compile procedures, packages, and functions (Y)
==> oracle-oci: STREAMS_CONFIGURATION import streams general metadata (Y)
==> oracle-oci: STREAMS_INSTANTIATION import streams instantiation metadata (N)
==> oracle-oci: DATA_ONLY import only data (N)
==> oracle-oci: VOLSIZE number of bytes in file on each volume of a file on tape
==> oracle-oci:
==> oracle-oci: The following keywords only apply to transportable tablespaces
==> oracle-oci: TRANSPORT_TABLESPACE import transportable tablespace metadata (N)
==> oracle-oci: TABLESPACES tablespaces to be transported into database
==> oracle-oci: DATAFILES datafiles to be transported into database
==> oracle-oci: TTS_OWNERS users that own data in the transportable tablespace set
==> oracle-oci:
==> oracle-oci: Import terminated successfully without warnings.
==> oracle-oci: Creating image from instance...
==> oracle-oci: Creating image from instance...
==> oracle-oci: Image created.
==> oracle-oci: Terminating instance (ocid1.instance.oc1.phx.anyhqljtu24ak7acd3vzqmhazefic7zo6bebsbqz3xvmnq7e23vqexiex2hq)...
==> oracle-oci: Terminated instance.
Build 'oracle-oci' finished.
Verify our Custom image is available
oci compute image list --compartment-id $compartment_ocid | jq -r '.data | map(select(."time-created" != null)) | sort_by(."time-created")[]| [."display-name",."operating-system",."time-created"]'
...
[
"CentOS-6.10-2020.02.24-0",
"CentOS",
"2020-02-24T20:57:32.240000+00:00"
]
[
"autonomous-instantclient-demo",
"Oracle Autonomous Linux",
"2020-03-05T11:02:39.026000+00:00"
]
Leave A Reply
You must be logged in to post a comment.
2 Comments
Great post, one suggestion is to add the dependency on the OCI CLI to make it work and an env var that points to the correct oci config entry, for example export access_cfg_file_account=TALLEN67
Hi Andy,
unfortunately the packer-oci builder is not able to use env variables to connect to specific tenancies, and does not support the authentication model used in the OCi cloud shell, yet