Knative Tutorial

In this chapter we will use Knative + Camel-k to deploy a simple route that will move *.xml file from one s3 bucket to another.

Enterprise Integration Pattern

Message Filter

The Message Filter has only a single output channel. If the message content matches the criteria specified by the Message Filter, the message is routed to the output channel. If the message content does not match the criteria, the message is discarded.

Demo Overview

The demo has an Apache Camel route configured as knative serving endpoint, that gets activated once it receives an event i.e. s3 object (xml file) name. Upon activation it starts a Apache Camel route that applies Message Filter EIP to process only the xml files. The matching xml files will be moved to another bucket called top(to process), while other files are discarded.

The following diagram shows the pictorial overview of the demo:

Message Filter

Deployment

Make s3 data bucket

  • minikube

  • minishift

S3_URL=$(minikube service -n knativetutorial minio-server --url)

S3_URL=$(minishift service -n knativetutorial minio-server --url)

For this demos we will be using a s3 bucket called data, run the following command to create the bucket in s3:

aws --endpoint-url $S3_URL s3 mb s3://data

Verify the created bucket by listing the s3 storage, running the following command:

aws --endpoint-url $S3_URL s3 ls s3://data

For more information on how to use aws cli with minio server is available here.

You can also create the bucket via the minio web console.

Make s3 to-process bucket

For this demo we will also be using a s3 bucket called top where the files will be moved after, run the following command to create the bucket in s3:

aws --endpoint-url $S3_URL s3 mb s3://top

Verify the created bucket by listing the s3 storage, running the following command:

aws --endpoint-url $S3_URL s3 ls s3://top

Deploy Cartoon Messages Mover

Navigate to camel-k demo sources folder:

cd $TUTORIAL_HOME/advanced/camel-k/quickstart

kamel run \
  --property 's3EndpointUrl=http://minio-server' \
  --property 'minioAccessKey=demoaccesskey' \
  --property 'minioSecretKey=demosecretkey' \
  src/main/java/CartoonMessagesMover.java

You can check the deployed integration context:

kamel get

Running the above command will return something like:

NAME                    PHASE                   CONTEXT
cartoon-messages-mover  Building Context        ctx-bhvn6lcba0587gdq5jbg

Watching Logs

Since a Cron job source is used in this section of the tutorial, it would emit events every minute. We can watch the logs of the service to see the messages delivered.

The logs could be watched using the command:

  • kubectl

  • oc

kubectl logs -n knativetutorial -f <pod-name> -c user-container
oc logs -n knativetutorial -f <pod-name> -c user-container
  • Using stern with the command stern -n knativetutorial cartoon-messages-mover, to filter the logs further add -c user-container to the stern command.

stern -n knativetutorial -c user-container cartoon-messages-mover

Test the Route

Upload test data

Navigate to the quickstart folder:

cd $TUTORIAL_HOME/advanced/camel-k/quickstart

  • minikube

  • minishift

S3_URL=$(minikube service -n knativetutorial minio-server --url)

S3_URL=$(minishift service -n knativetutorial minio-server --url)

Upload XML file called message1.xml to s3 bucket called data,

aws --endpoint-url $S3_URL s3 cp data/message1.xml  s3://data

Upload a text file called hello.txt to s3 bucket called data,

aws --endpoint-url $S3_URL s3 cp data/hello.txt  s3://data

Verify if the XML and text files were uploaded to the s3 bucket:

aws --endpoint-url $S3_URL s3 ls s3://data

Invoke service

Invoke the service by passing a header fileName with value message1.xml, this fileName header value will be used by Camel route to move the file from s3 bucket data to top.

  • kubectl

  • oc

IP_ADDRESS="$(minikube ip):$(kubectl get svc istio-ingressgateway --namespace istio-system --output 'jsonpath={.spec.ports[?(@.port==80)].nodePort}')"

IP_ADDRESS="$(minishift ip):$(oc get svc istio-ingressgateway --namespace istio-system --output 'jsonpath={.spec.ports[?(@.port==80)].nodePort}')"

We need to know the knative service cartoon-messages-mover domain name that needs to be passed as Host header, we will query and and set that value in an environment variable called SVC_URL:

  • kubectl

  • oc

SVC_URL=`kubectl -n knativetutorial get ksvc cartoon-messages-mover -ojsonpath='{.status.domain}'`

SVC_URL=`oc  -n knativetutorial get ksvc cartoon-messages-mover -ojsonpath='{.status.domain}'`

curl -v -H "Host:${SVC_URL}" -H 'fileName: message1.xml' $IP_ADDRESS

The event simulation above will be removed once the CamelSource event source is available with Knative Eventing.

On success which is usually HTTP 200, you can list the out top bucket to see the moved message1.xml file.

aws --endpoint-url $S3_URL s3 ls s3://top

Lets now try sending the request for hello.txt:

curl -v -H "Host:${SVC_URL}" -H 'fileName: hello.txt' $IP_ADDRESS

Since we process only *.xml files the top bucket will not have the file hello.txt moved from data bucket, you can verify the same using the command:

aws --endpoint-url $S3_URL s3 ls s3://top

When the cartoon-messages-mover has been scaled to zero; when giving a new request it will try to scale up, it will not be able to serve the request until the pod is up, during such time you might get HTTP 502 Bad Gateway errors, give some more time for the pod to come up before issuing the request again.

Cleanup

kamel delete cartoon-messages-mover -n knativetutorial