Pre-packaged Applications

The Spring team provides and supports a selection of pre-packaged applications used to assemble various data integration and processing pipelines and to support Spring Cloud Data Flow development, learning and experimentation.

Getting Started

All pre-packaged streaming applications:

  • Are available as Apache Maven artifacts or Docker images
  • Use RabbitMQ or Apache Kafka
  • Support monitoring through Prometheus and InfluxDB
  • Contain metadata for application properties used in the UI and code completion in the shell

You can register stream and task applications by using the Data Flow UI or the shell.

You can register applications individually by using the app register command or in bulk by using the app import command.

For streams, depending on whether you use Kafka or RabbitMQ, you can register the applications by using the respective URLs:

For tasks, you can use the following URL:

Using the Data Flow UI, these links are included for pre-fill:

Bulk register applications

From the Data Flow Shell, you can bulk import and register the applications, as the following example shows:

dataflow:>app import --uri https://dataflow.spring.io/kafka-maven-latest

The latest bulk-registration links are updated as part of the release process. There are additional bulk-registration links to pre-packaged applications for a specific release.

Stream Applications

The Spring team develops and maintains stream applications and publishes these applications to the Spring public Maven repository and to Dockerhub in accordance with a release schedule, normally following significant Spring Boot or Spring Cloud Stream releases. The pre-packaged stream applications are Spring Boot executable jars that are built with a specific binder implementation. For each stream app, we provide separate exacutable applications for RabbitMQ and Kafka.

Currently available stream applications include:

Source Processor Sink
cdc-debezium aggregator cassandra
file bridge counter
ftp counter file
gemfire filter ftp
gemfire-cq groovy-filter gemfire
http groovy-transform hdfs
jdbc grpc jdbc
jms header-enricher log
load-generator httpclient mongodb
loggregator image-recognition mqtt
mail object-detection pgcopy
mongodb pmml rabbit
mqtt pose-estimation redis-pubsub
rabbit python-http router
s3 python-jython s3
sftp scriptable-transform sftp
sftp-dataflow splitter task-launcher-dataflow
syslog tasklaunchrequest-transform tcp
tcp tcp-client throughput
tcp-client tensorflow websocket
time transform
trigger twitter-sentiment
triggertask
twitterstream

Task Applications

The Spring team develops and maintains task/batch applications and publishes these applications to the Spring public Maven repository and to Dockerhub in accordance with a planned release schedule, normally following significant Spring Boot, Spring Cloud Task, or Spring Batch releases.

Currently available task/batch applications include the following:

timestamp
timestamp-batch

Bulk Registration of Stream Applications

Spring Cloud Data Flow supports bulk registration of applications, via a standard properties file format. For convenience, we publish static properties files with application-URIs (for either maven or docker) for all the out-of-the-box stream and task/batch apps. You can use these files in Spring Cloud Data Flow to register all the application-URIs in bulk. Alternately, you can register them individually or provide your own custom property file with only the required application-URIs in it. Bulk registration is convenient for getting started with SCDF. To reduce clutter, it is recommended, to maintain a “focused” list of desired application-URIs in a custom property file.

Supported Spring Cloud Stream Applications

Artifact Type Latest Stable Release SNAPSHOT Release
spring-cloud-stream 2.1.x
spring-boot 2.1.x
RabbitMQ + Maven https://dataflow.spring.io/rabbitmq-maven-latest https://dataflow.spring.io/Einstein-BUILD-SNAPSHOT-stream-applications-rabbit-maven
RabbitMQ + Docker https://dataflow.spring.io/rabbitmq-docker-latest https://dataflow.spring.io/Einstein-BUILD-SNAPSHOT-stream-applications-rabbit-docker
Kafka + Maven https://dataflow.spring.io/kafka-maven-latest https://dataflow.spring.io/Einstein-BUILD-SNAPSHOT-stream-applications-kafka-maven
Kafka + Docker https://dataflow.spring.io/kafka-docker-latest https://dataflow.spring.io/Einstein-BUILD-SNAPSHOT-stream-applications-kafka-docker

Supported Spring Cloud Task/Batch Applications

Artifact Type Latest Stable Release SNAPSHOT Release
spring-cloud-stream 2.1.x
spring-boot 2.1.x
Maven https://dataflow.spring.io/task-maven-latest https://dataflow.spring.io/Elston-BUILD-SNAPSHOT-task-applications-maven
Docker https://dataflow.spring.io/task-docker-latest https://dataflow.spring.io/Elston-BUILD-SNAPSHOT-task-applications-docker

The following table lists previous releases for reference only. NOTE: These may no longer be supported (i.e. they depend on an EOL spring-boot release):

Previous Releases of Stream Applications

Artifact Type Previous Release
spring-cloud-stream 2.0.x
spring-boot 2.0.x
RabbitMQ + Maven https://dataflow.spring.io/Darwin-SR3-stream-applications-rabbit-maven
RabbitMQ + Docker https://dataflow.spring.io/Darwin-SR3-stream-applications-rabbit-docker
Kafka + Maven https://dataflow.spring.io/Darwin-SR3-stream-applications-kafka-maven
Kafka + Docker https://dataflow.spring.io/Darwin-SR3-stream-applications-kafka-docker

Previous Releases of Spring Cloud Task/Batch Applications

Artifact Type Previous Release
spring-cloud-stream 2.0.x
spring-boot 2.0.x
Maven https://dataflow.spring.io/Dearborn-SR1-task-applications-maven
Docker https://dataflow.spring.io/Dearborn-SR1-task-applications-docker