confluentinc/cp-flink-models repository overview

⁠Confluent Platform for Apache Flink time series model server

The model server that the AI_DETECT_ANOMALIES and AI_FORECAST functions of Confluent Platform for Apache Flink call. Each server runs one time series model and answers the functions' requests with a forecast of the next values and a range for each. This image is part of a preview.

⁠Models

The MODEL environment variable picks the model, by the name the functions' model setting uses:

⁠Tags

• <version>: for amd64 and arm64.
• <version>-amd64 and <version>-arm64: for one architecture each.

The image is based on Red Hat UBI 10, so on amd64 it runs only on CPUs of the x86-64-v3 level, which have AVX2.

⁠Run

docker run -e MODEL=ttm -p 8080:8080 confluentinc/cp-flink-models:<version>

GET /readyz returns 200 once the model is loaded, and the functions call POST /v1/forecast. A server that cannot load its model exits.

In Confluent Manager for Apache Flink, set each model's URL, such as http://<host>:8080/v1/forecast, in the environment's statement defaults as ai.models.<model>.endpoint, in both the interactive and detached sections. A function call then names only the model:

AI_DETECT_ANOMALIES(reading, reading_time, JSON_OBJECT('model' VALUE 'ttm'))
  OVER (ORDER BY reading_time RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW)

Give the container a CPU limit, such as --cpus 4 or a Kubernetes resources.limits.cpu. Without one, the server runs a thread per CPU of the machine, and requests that arrive together slow each other down.

⁠Model files

At start-up the server downloads its model from Hugging Face, at the commit its image version pins, and logs Downloading model=<model> from <repo>@<commit>. PatchTST-FM is about 1 GB. Mount a writable volume at /home/appuser/.cache/huggingface so that a restart doesn't download it again.

Without internet access, mount a directory holding the model's config.json and model.safetensors at /mnt/models, downloaded at the commit the server logs. To see that commit, start the image once without the mount and without a network, such as with docker run --rm --network none -e MODEL=ttm confluentinc/cp-flink-models:<version>: it logs the line above, then exits. For example, for TTM:

hf download ibm-granite/granite-timeseries-ttm-r3 config.json model.safetensors --revision <commit> --local-dir ttm
docker run -e MODEL=ttm -v "$PWD/ttm:/mnt/models:ro" -p 8080:8080 confluentinc/cp-flink-models:<version>

The server then exits at start-up, instead of downloading, if either file is missing or the files are for another model or cannot be loaded.

⁠Security

The endpoint has no authentication or TLS, so run the server where only your Flink jobs can reach it, for example in their Kubernetes cluster behind a network policy. The server runs as a non-root user, UID 1000, or as the arbitrary UID in group 0 that OpenShift assigns. It sends no usage reports.

⁠License

The Confluent License Agreement is in /licenses in the image. The models are IBM's, and their licenses are on their Hugging Face pages.