How to retrieve an OCI Marketplace image id using advanced functionality in the OCI CLI utility

If you ever need to deploy an OCI Marketplace image using your automation, the information that you will have to feed to your script will be:

  "listingId": "ocid1.appcataloglisting.oc1..aaaaaaaam7ewzrjbltqiarxukuk72v2lqkdtpqtwxqpszqqvrm7likfnpt5q",
  "listingResourceId": "ocid1.image.oc1..aaaaaaaaa5u7qtj3j3um4zryg3qkn6if2sqctqjfh46pdq5z56kq6zagg4va",

Let’s say we need to find this information for a specific Marketplace image, like RocketChat:

We can use the OCI cli to retrieve the details needed by terraform, but since there is no direct command to fetch information about a Marketplace listing we are going to use the ‘raw-request’ functionality that the cli provides as a fallback to access the OCI API when needed for advanced use cases
I have created a gist that uses the OCI cli to perform this kind of search, you can find it on GitHub

#!/bin/bash
#Example string: 'FortiGate Next-Gen Firewall (BYOL)'
export STERM="$1"
#Example version: '6.4.0'
export SVERSION="$2"
export REGION="uk-london-1"
export LID=`oci raw-request --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings --http-method GET | jq --arg STERM "$STERM" '.data[] | select ( .displayName | contains($STERM))' | jq -r '.listingId'`
oci raw-request --http-method GET --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings/$LID/resourceVersions | jq --arg SVERSION "$SVERSION" '.data[] | select ( .listingResourceVersion | contains($SVERSION))'

The script accepts two parameters

• the Marketplace image name
• the Image version needed

mrossi@cloudshell:~ (us-phoenix-1)$ ./getMarketplaceImageId 'RocketChat' '2.4.9'
{
"listingId": "ocid1.appcataloglisting.oc1..aaaaaaaapjvcl7yie67z6ag4hwfgpiegwpy5izkaaq3p3rngvmtmqm7zcaxq",
"listingResourceId": "ocid1.image.oc1..aaaaaaaajq2pfkmd6nl23ouho2ulxldoy2or5zy7dbiv46qhu6tsqm6ftila",
"listingResourceVersion": "2.4.9",
"timePublished": "2020-03-05T13:27:11.940Z"
}

Let’s break down the script:

export STERM="$1"
#6.4.0
export SVERSION="$2"
export REGION="uk-london-1"

These are initialisation parameters, assigning the search parameters from the script input params, and setting a default OCI region for the API queries

export LID=`oci raw-request --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings --http-method GET | jq --arg STERM "$STERM" '.data[] | select ( .displayName | contains($STERM))' | jq -r '.listingId'`

The first call to the OCI cli retrieves the Listing id for a given Marketplace image:

mrossi@cloudshell:~ (us-phoenix-1)$ oci raw-request --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings --http-method GET | jq --arg STERM "$STERM" '.data[] | select ( .displayName | contains($STERM))' | jq -r '.listingId'
ocid1.appcataloglisting.oc1..aaaaaaaapjvcl7yie67z6ag4hwfgpiegwpy5izkaaq3p3rngvmtmqm7zcaxq

Let’s break down the command:
The first part retrieves all the Marketplace listings for a given region:

mrossi@cloudshell:~ (us-phoenix-1)$ oci raw-request --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings --http-method GET
{
  "data": [
    {
      "displayName": "0Chain Data Privacy Compliance Platform",
      "listingId": "ocid1.appcataloglisting.oc1..aaaaaaaahfwb7mtjpi45wc4waduujja5fdeslfio4r2amuyuz3x62x5rxina",
      "publisherName": "0Chain LLC",
      "summary": "Add data privacy, protection, and private sharing to your business"
    },
....
    {
      "displayName": "RocketChat",
      "listingId": "ocid1.appcataloglisting.oc1..aaaaaaaapjvcl7yie67z6ag4hwfgpiegwpy5izkaaq3p3rngvmtmqm7zcaxq",
      "publisherName": "Rocket.Chat",
      "summary": "The ultimate Open Source communication platform"
    },
    {
      "displayName": "SUSE Linux Enterprise Server 12 SP4 (BYOS)",
      "listingId": "ocid1.appcataloglisting.oc1..aaaaaaaa3mflvqev6iw4jnjibtrlgdz2h75kpmdwgvhifljffv5b5xbcqmxq",
      "publisherName": "SUSE",
      "summary": "SLES 12 SP4"
    },
    {
      "displayName": "SUSE Linux Enterprise Server 12 SP4 (BYOS)",
      "listingId": "ocid1.appcataloglisting.oc1..aaaaaaaaowy672zei2w2dy3s2z4c7kytbrd55cnvox3djurgswguiang32ga",
      "publisherName": "SUSE",
      "summary": "SLES 12 SP4"
    },
 ...
  ],
  "headers": {
    "Connection": "keep-alive",
    "Content-Type": "application/json",
    "Date": "Tue, 19 May 2020 13:31:54 GMT",
    "Transfer-Encoding": "chunked",
    "Vary": "Accept-Encoding",
    "X-Content-Type-Options": "nosniff",
    "opc-next-page": "AAAAAAAAAAAAFCCBIFAUCQKBIFAUCQKHOVUVIY2NOFPTETJNMV5HQX3KKJKHE2DOHBZUWRTRNMZWWODWJIYHM3BTOE3VSVLTNJEGUSCPIVGVM5TGOM2GC3LCFVDHMTLCLA2XA6SXJVCEMRSNNNIHQVCPLFVGSU3DIVIGKTTZI5UE24LUI5KHQLKFOFVVKVLNO5UWS2KQOVWGM4LCOA3HQSLXOI4FATJWLFITAM2DIEZDK3KXMN3C233IJFLEGZZVNQYVATKZJAWUY5DPGFIUUWBVHBZWKSTOOFPUIUKGO5VTCSRZINBVE4JNKI4HKTTZN5SFO4CJOBAUUTDUKVSDKUCXNVVV6ZBTIZMVGYSVKFEHC4BYNJUHQOJQMNGWI6CNGAZFMSTJMFMEUQSCNV3VEZ32NFGWQVCGJFYDOLJTJBXUCVKYM5JWYYKGPFDWC5BSGF2HIZ27KFIEMXZQKVDEUUD2KRHGUVKNJNTFETKHIZGDEWTYGMYHG23CJNSGGMCEINXFKY2XJJWVG4JRGBKGOWSIKFUXKSCCKM3XAZ2EIJHWETLOPFXU23KRNA4UMNLOIQ2FU53CKRUEKWSYGFPUINSVJRWW63DRKVKUKQZVIF5FMSKMOZXG6WS7N5YF62LQMEZE4MLLLBRWQUCDOZUXA23WMJQTGODRJJHG62ZSKZ5HO4ZWGFKWQSLWHFSFQYLCMQYFINRUJRGVAWBWJA3UEVCYKAYFQ6TTJNYVKU2BGI4WSQLXMJDFUV2GPBLGIMDZM5QWY2KMNJTXQ6LCNBPWSSDUOIZE643IL5VG6SD2HBLDKSS2JVVHEVLULBQVE6TMOZWWSN3YJFKWI2TXMRQXQMLWJ5IXCNDSI5LS233SJI4WKSJXIRGTS2RQHFBXGNBVINRWUOCWLFBG26BVGVEECMZZIRDDQYLGJN3FS3DUNQYWGWLLNFHGYVTRONNFK2K2PF2XGT2JMRVWSZ3LG5LFCY2HORMXOODXOREEO3ZZLJVVU6CIKNUGK42DKJLEQTCMKEYXC3DFPBAWMQSNKZXTE5ZTFVHQ",
    "opc-request-id": "843C1ADDCBC74192AFA0499764567CD8/3349DFC98814A8F5F33B1F767B7A0FE6/7D4C870DF934CD46CD94537C86F0969D"
  },
  "status": "200 OK"
}

The second part filters the global result json object and retrieves only information related to the Image we’re interested in:

oci raw-request --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings --http-method GET | jq --arg STERM "$STERM" '.data[] | select ( .displayName | contains($STERM))'

I am using the jq utility to do that by setting up a filter on the displayName field:

jq --arg STERM "$STERM" '.data[] | select ( .displayName | contains($STERM))'

mrossi@cloudshell:~ (us-phoenix-1)$ oci raw-request --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings --http-method GET | jq --arg STERM "$STERM" '.data[] | select ( .displayName | contains($STERM))'
{
"displayName": "RocketChat",
"listingId": "ocid1.appcataloglisting.oc1..aaaaaaaapjvcl7yie67z6ag4hwfgpiegwpy5izkaaq3p3rngvmtmqm7zcaxq",
"publisherName": "Rocket.Chat",
"summary": "The ultimate Open Source communication platform"
}

The third part filters the single listing result json object and retrieves the listingId field, again using the jq utility and assigns it to an utility variable

export LID=`oci raw-request --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings --http-method GET | jq --arg STERM "$STERM" '.data[] | select ( .displayName | contains($STERM))' | jq -r '.listingId'`

Using the collected listing ID we can retrieve the actual information we’re interested in by issuing another API call:

oci raw-request --http-method GET --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings/$LID/resourceVersions | jq --arg SVERSION "$SVERSION" '.data[] | select ( .listingResourceVersion | contains($SVERSION))'

This command filters the API call output using jq to filter only the specific information we’re interested in:

mrossi@cloudshell:~ (us-phoenix-1)$ oci raw-request --http-method GET --target-uri https://iaas.$REGION.oraclecloud.com/20160918/appCatalogListings/$LID/resourceVersions | jq --arg SVERSION "$SVERSION" '.data[] | select ( .listingResourceVersion | contains($SVERSION))'
{
"listingId": "ocid1.appcataloglisting.oc1..aaaaaaaapjvcl7yie67z6ag4hwfgpiegwpy5izkaaq3p3rngvmtmqm7zcaxq",
"listingResourceId": "ocid1.image.oc1..aaaaaaaajq2pfkmd6nl23ouho2ulxldoy2or5zy7dbiv46qhu6tsqm6ftila",
"listingResourceVersion": "2.4.9",
"timePublished": "2020-03-05T13:27:11.940Z"
}

This technique of doing RAW api calls can be extended for any use that the standard cli commands do not cover, while the filtering through jq can be used for manipulating any json output. The OCI cli has an internal filtering mechanism that doesn’t play well with the output from the raw request subcommand
Have fun !
Mattia