Knative Tutorial

In this chapter we will use Knative + Camel-k to deploy a route that will parse the content of the xml file that was added to an s3 bucket and routes the message to different endpoints based on the content.

Content Based Router

The Content-Based Router examines the message content and routes the message onto a different channel based on data contained in the message. The routing can be based on a number of criteria such as existence of fields, specific field values etc. When implementing a Content-Based Router, special caution should be taken to make the routing function easy to maintain as the router can become a point of frequent maintenance. In more sophisticated integration scenarios, the Content-Based Router can take on the form of a configurable rules engine that computes the destination channel based on a set of configurable rules.

Demo Overview

The demo has an Apache Camel route configured as knative serving endpoint, that gets activated once it receives an event i.e. s3 object (file) name. Upon activation it starts a Apache Camel route that applies Content Based Router EIP based on cartoon’s genre. All cartoons of genre comedy will be delivered to a knative channel called genre-comedy and other genres wil be delivered to knative channel called genre-others.

There will be two message handlers that will be deployed to handle the respective genre messages:

The following diagram shows the pictorial overview of the demo:

Content Based Routing

Deployment

You can skip this step and move to create knative channels if you have already completed the previous section

Make s3 data bucket

  • minikube

  • minishift

S3_URL=$(minikube service -n knativetutorial minio-server --url)

S3_URL=$(minishift service -n knativetutorial minio-server --url)

For this demo we will be using a s3 bucket called data, run the following command to create the bucket in s3:

aws --endpoint-url $S3_URL s3 mb s3://data

Verify the created bucket by listing the s3 storage, running the following command:

aws --endpoint-url $S3_URL s3 ls s3://data

For more information on how to use aws cli with minio server is available here.

You can also create the bucket via the minio web console.

Create Knative Channels

Navigate to camel-k demo sources folder:

cd $TUTORIAL_HOME/advanced/camel-k/quickstart

  • kubectl

  • oc

kubectl  -n knativetutorial apply -f channels.yaml

Checking the channels created:

kubectl  -n knativetutorial get channels.eventing.knative.dev

Running the above commands should produce output as shown:

NAME           READY   REASON
genre-comedy   True
genre-others   True
oc  -n knativetutorial apply -f channels.yaml

Checking the channels created:

oc  -n knativetutorial get channels.eventing.knative.dev

Running the above commands should produce output as shown:

NAME           READY   REASON
genre-comedy   True
genre-others   True

Deploy Routes

Navigate to the quickstart folder:

cd $TUTORIAL_HOME/advanced/camel-k/quickstart

Deploy Comedy Genre Handler

kamel run \
  --property 's3EndpointUrl=http://minio-server' \
  --property 'minioAccessKey=demoaccesskey' \
  --property 'minioSecretKey=demosecretkey' \
 src/main/java/ComedyGenreHandler.java

Note the first build usually takes time, you can watch the status with commands
  • kubectl

  • oc

kubectl -n knativetutorial get integrations.camel.apache.org comedy-genre-handler

oc -n knativetutorial get integrations.camel.apache.org comedy-genre-handler

A successfully built integration context will be in a phase called Running as shown below:

kamel get
# the command will return
NAME                   PHASE   CONTEXT
comedy-genre-handler    Running ctx-bi0h1ur6a8efvn05oqbg

Deploy Others Genre Handler

kamel run \
  --property 's3EndpointUrl=http://minio-server' \
  --property 'minioAccessKey=demoaccesskey' \
  --property 'minioSecretKey=demosecretkey' \
 src/main/java/OthersGenreHandler.java

Note the first build usually takes time, you can watch the status with commands
  • kubectl

  • oc

kubectl -n knativetutorial get integrations.camel.apache.org others-genre-handler

oc -n knativetutorial get integrations.camel.apache.org others-genre-handler

A successfully built integration context will be in a phase called Running as shown below:

kamel get
# the command will return
NAME                    PHASE   CONTEXT
comedy-genre-handler    Running ctx-bi0h1ur6a8efvn05oqbg
comedy-genre-handler    Running ctx-bi0h1ur6a8efvn05oqbg

Deploy Cartoon Genre Router

kamel run \
  --property 's3EndpointUrl=http://minio-server' \
  --property 'minioAccessKey=demoaccesskey' \
  --property 'minioSecretKey=demosecretkey' \
 src/main/java/CartoonGenreRouter.java

Note the first build usually takes time, you can watch the status with command:
  • kubectl

  • oc

kubectl -n knativetutorial get integrations.camel.apache.org cartooon-genre-router

oc -n knativetutorial get integrations.camel.apache.org cartooon-genre-router

A successfully built integration context will be in a phase called Running as shown below:

kamel get
# the command will return
NAME                    PHASE   CONTEXT
cartoon-genre-router    Running ctx-bi0h1ur6a8efvn05oqbg
comedy-genre-handler    Running ctx-bi0h1ur6a8efvn05oqbg
others-genre-handler    Running ctx-bi0h1ur6a8efvn05oqbg

See what you have deployed

The command above will create the following Kubernetes resources

Some of the columns has removed from the above output for brevity

Services

  • kubectl

  • oc

kubectl -n knativetutorial  get svc

oc -n knativetutorial  get svc

The above command should return you the following services:

NAME                                 TYPE           CLUSTER-IP       EXTERNAL-IP                                           PORT(S)           AGE
camel-k-operator                     ClusterIP      10.108.197.67    <none>                                                60000/TCP         82m
cartoon-genre-router-zx66c-service   ClusterIP      10.101.214.254   <none>                                                80/TCP,9090/TCP   8m59s
comedy-genre-handler                 ExternalName   <none>           istio-ingressgateway.istio-system.svc.cluster.local   <none>            32m
comedy-genre-handler-rgrt8-service   ClusterIP      10.111.249.8     <none>                                                80/TCP,9090/TCP   33m
genre-comedy-channel-kxcwc           ClusterIP      10.97.30.253     <none>                                                80/TCP            61m
genre-others-channel-l7kxj           ClusterIP      10.111.225.184   <none>                                                80/TCP            61m
minio-server                         NodePort       10.99.84.238     <none>                                                80:32385/TCP      84m
others-genre-handler                 ExternalName   <none>           istio-ingressgateway.istio-system.svc.cluster.local   <none>            32m
others-genre-handler-jnwsd-service   ClusterIP      10.111.172.10    <none>                                                80/TCP,9090/TCP   32m

Integration Contexts

  • kubectl

  • oc

kubectl -n knativetutorial  get integrations.camel.apache.org

oc -n knativetutorial  get get integrations.camel.apache.org

NAME                    PHASE     CONTEXT
cartoon-genre-router    Running ctx-bi0h1ur6a8efvn05oqbg
comedy-genre-handler    Running ctx-bi0h1ur6a8efvn05oqbg
others-genre-handler    Running ctx-bi0h1ur6a8efvn05oqbg

Deployments

  • kubectl

  • oc

kubectl -n knativetutorial  get deployments

oc -n knativetutorial  get get deployments

NAME                                    DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
camel-k-operator                        1         1         1            1           93m
cartoon-genre-router-zx66c-deployment   0         0         0            0           19m
comedy-genre-handler-rgrt8-deployment   0         0         0            0           43m
minio-server                            1         1         1            1           94m
others-genre-handler-jnwsd-deployment   0         0         0            0           43m

Watching logs

Watching the logs of the deployed camel route.

Comedy Genre Handler

Since a Cron job source is used in this section of the tutorial, it would emit events every minute. We can watch the logs of the service to see the messages delivered.

The logs could be watched using the command:

  • kubectl

  • oc

kubectl logs -n knativetutorial -f <pod-name> -c user-container
oc logs -n knativetutorial -f <pod-name> -c user-container
  • Using stern with the command stern -n knativetutorial comedy-genre-handler, to filter the logs further add -c user-container to the stern command.

stern -n knativetutorial -c user-container comedy-genre-handler

Other Genre Handler

Since a Cron job source is used in this section of the tutorial, it would emit events every minute. We can watch the logs of the service to see the messages delivered.

The logs could be watched using the command:

  • kubectl

  • oc

kubectl logs -n knativetutorial -f <pod-name> -c user-container
oc logs -n knativetutorial -f <pod-name> -c user-container
  • Using stern with the command stern -n knativetutorial others-genre-handler, to filter the logs further add -c user-container to the stern command.

stern -n knativetutorial -c user-container others-genre-handler

Comedy Genre Router

Since a Cron job source is used in this section of the tutorial, it would emit events every minute. We can watch the logs of the service to see the messages delivered.

The logs could be watched using the command:

  • kubectl

  • oc

kubectl logs -n knativetutorial -f <pod-name> -c user-container
oc logs -n knativetutorial -f <pod-name> -c user-container
  • Using stern with the command stern -n knativetutorial comedy-genre-router, to filter the logs further add -c user-container to the stern command.

stern -n knativetutorial -c user-container comedy-genre-router

Test the Route

Upload test data

Navigate to the quickstart folder:

cd $TUTORIAL_HOME/advanced/camel-k/quickstart

  • minikube

  • minishift

S3_URL=$(minikube service -n knativetutorial minio-server --url)

S3_URL=$(minishift service -n knativetutorial minio-server --url)

Upload XML file called message1.xml to s3 bucket called data,

aws --endpoint-url $S3_URL s3 cp data/message1.xml  s3://data

Upload XML file called message2.xml to s3 bucket called data,

aws --endpoint-url $S3_URL s3 cp data/message2.xml  s3://data

Verify if the XML files were uploaded to the s3 bucket:

aws --endpoint-url $S3_URL s3 ls s3://data

Invoke service

Invoke the service by passing a header fileName with value message1.xml, this fileName header value will be used by Camel route to process the body content of the file to move them to genre-comedy or genre-others content buckets in s3.

  • kubectl

  • oc

IP_ADDRESS="$(minikube ip):$(kubectl get svc istio-ingressgateway --namespace istio-system --output 'jsonpath={.spec.ports[?(@.port==80)].nodePort}')"

IP_ADDRESS="$(minishift ip):$(oc get svc istio-ingressgateway --namespace istio-system --output 'jsonpath={.spec.ports[?(@.port==80)].nodePort}')"

We need to know the knative service cartoon-genre-router domain name that needs to be passed as Host header, we will query and and set that value in an environment variable called SVC_URL:

  • kubectl

  • oc

SVC_URL=`kubectl -n knativetutorial get ksvc cartoon-genre-router -ojsonpath='{.status.domain}'`

SVC_URL=`oc  -n knativetutorial get ksvc cartoon-genre-router -ojsonpath='{.status.domain}'`

curl -v -H "Host:${SVC_URL}" -H 'fileName: message1.xml' $IP_ADDRESS

The event simulation above will be removed once the CamelSource event source is available with Knative Eventing.

On success which is usually HTTP 200, you can list the out genre-comedy bucket to see the file tomandjerry.xml. You can list the bucket using the command:

aws --endpoint-url $S3_URL s3 ls s3://genre-comedy

Lets try to upload non comedy genre to see the file getting uploaded to genre-others,

curl -v -H "Host:${SVC_URL}" -H 'fileName: message2.xml' $IP_ADDRESS

Like before on success which is usually HTTP 200, you can list the out genre-others bucket to see the file theadventuresoftenaliraman.xml. You can list the bucket using the command:

aws --endpoint-url $S3_URL s3 ls s3://genre-others

When the cartoon-genre-router has been scaled to zero; when giving a new request it will try to scale up, it will not be able to serve the request until the pod is up, during such time you might get HTTP 502 Bad Gateway errors, give some more time for the pod to come up before issuing the request again.

Right now camel-k does not add Liveliness and Readiness Probes which can help avoid this issue. There is a open issue which is fixed and will be part of 0.3.1 release.

Cleanup

kamel delete comedy-genre-handler -n knativetutorial && \
kamel delete others-genre-handler -n knativetutorial && \
kamel delete cartoon-genre-router -n knativetutorial && \
cd $TUTORIAL_HOME/advanced/camel-k/knative

  • kubectl

  • oc

kubectl delete -f channels.yaml -n knativetutorial && \
cd $TUTORIAL_HOME

oc delete -f channels.yaml -n knativetutorial && \
cd $TUTORIAL_HOME