OCI Streaming with Terraform and Functions: Build Your First Producer
Introduction
In this article we’ll focus on OCI Streaming Terraform to show how easy it is to build your first producer function. Oracle Cloud Infrastructure (OCI) Streaming is a Kafka-compatible, fully managed messaging service designed for real-time data pipelines and event-driven architectures. Instead of managing your own Kafka cluster, you can spin up a stream in OCI, publish messages from your applications, and consume them with minimal effort.
In this post, we’ll use Terraform to provision OCI Streaming and then build a serverless producer function that pushes events into the stream. This is just a starting point — the full architecture (including consumer functions, Autonomous Database integration, and Notifications) is covered in Lesson 6 of my OCI Serverless Functions Course.
Figure 1. High-level architecture of OCI Streaming with Terraform and Functions.
Provisioning OCI Streaming with Terraform
Let’s start by creating a Stream Pool and a Stream:
resource "oci_streaming_stream_pool" "FoggyKitchenStreamPool" {
compartment_id = var.compartment_ocid
name = "FoggyKitchenStreamPool"
}
resource "oci_streaming_stream" "FoggyKitchenStream" {
name = "FoggyKitchenStream"
partitions = 1
retention_in_hours = 24
stream_pool_id = oci_streaming_stream_pool.FoggyKitchenStreamPool.id
}
Add outputs for easier testing:
output "stream_id" {
value = oci_streaming_stream.FoggyKitchenStream.id
}
Building the Producer Function
A simple OCI Function can publish messages into our stream:
import io, json, oci
from fdk import response
def handler(ctx, data: io.BytesIO = None):
cfg = oci.config.from_file()
streaming_client = oci.streaming.StreamClient(cfg)
put_message = oci.streaming.models.PutMessagesDetailsEntry(
key="foggykitchen-key".encode('utf-8'),
value="Hello from OCI Function!".encode('utf-8')
)
messages = oci.streaming.models.PutMessagesDetails(messages=[put_message])
result = streaming_client.put_messages(
stream_id="",
put_messages_details=messages
)
return response.Response(
ctx,
response_data=json.dumps({"status": "Message sent", "result": str(result)}),
headers={"Content-Type": "application/json"}
)
Fast Deployment with Terraform Module
Instead of manually building and deploying the function, you can use my open-source Terraform module:
module "oci-fk-initiator-function" {
source = "github.com/mlinxfeld/terraform-oci-fk-function"
compartment_ocid = var.compartment_ocid
fk_fn_name = "fninitiator"
func_py_content = data.local_file.fninitiator_func_py.content
fn_config = {
"STREAM_OCID" = oci_streaming_stream.FoggyKitchenStream.id,
"STREAM_ENDPOINT" = data.oci_streaming_stream_pool.FoggyKitchenStreamPool.endpoint_fqdn
}
}
⚠️ This is a simplified snippet. In my course, I cover the full deployment with API Gateway triggers, OCIR authentication, and IAM policies.
👉 Explore the module here: terraform-oci-fk-function.
👉 See the training examples here: training/.
Testing with OCI CLI
You can check the message with the CLI:
oci streaming stream message get \
--stream-id \
--partition 0
Before You Start
Make sure your IAM configuration is in place — the entire producer workflow relies on proper Dynamic Groups and policies.
See OCI IAM Policies Terraform Example to learn how Functions securely access Streaming and other services without user credentials.
Wrapping Up
In this post, we’ve built the producer side of an event-driven architecture on Oracle Cloud Infrastructure. With Terraform we provisioned a Stream Pool and a Stream, and then deployed a serverless function that pushes messages into the stream. We also verified the flow using the OCI CLI.
This is the foundation of real-time pipelines — producers send events into OCI Streaming where they can be consumed by other services, functions, or databases.
In the next article, we’ll extend this architecture with a consumer function that reads events from the stream and stores them inside Autonomous Database (ADB), enabling real-time analytics and dashboards.
👉 The complete scenario — including three functions, API Gateway integration, Notifications, Streaming, and ADB — is explained step by step in Lesson 6 of my OCI Serverless Functions Course. It’s a hands-on lab where you’ll learn how to automate everything with Terraform/OpenTofu and test the entire pipeline end-to-end.
🚀 Take the Next Step with OCI Functions
Learn how to build the full event-driven pipeline:
API Gateway → Functions → Streaming → Autonomous Database.
All explained step by step in Lesson 6 of my OCI Serverless Functions Course.
🔒 Lifetime • ⏱️ Self-paced • 🧪 Real labs
Check also other courses:​
How to automate Oracle Cloud Infrastructure with Terraform/OpenTofu (2025 Edition)
