Seldon Core

Seldon Core 2 is a Kubernetes-native MLOps and LLMOps framework for deploying machine learning models and Large Language Model systems at scale.

Use it when

• •You need a Kubernetes-native solution for ML model deployment.
• •You're deploying both traditional ML models and Large Language Models (LLMs).
• •You require advanced MLOps features like A/B testing, canary deployments, and experiment routing.
• •Your infrastructure spans multiple environments (on-premise, hybrid, multi-cloud).
• •You need sophisticated monitoring and observability for ML systems with auditable prediction data.
• •You want to reduce infrastructure costs through multi-model serving and resource optimization.
• •You have a team with strong Kubernetes and MLOps expertise.
• •You need to compose complex AI applications through pipelines.

Watch out

• ⚠Auto-scaling limitations: Requires additional setup (like KEDA) and does not support scaling to zero when idle.
• ⚠Kubernetes expertise required: Not suitable for companies with limited MLOps capabilities.
• ⚠Community support: Only average community support available compared to more established platforms.
• ⚠Configuration complexity: Common deployment issues include incorrect configuration settings and accessibility problems.
• ⚠Local model deployment issues: Problems with syncing/copying local models to persistent volumes.
• ⚠Network timing issues: Istio VirtualServices may not be ready immediately after container startup.
• ⚠Metrics integration challenges: Issues with integrating Triton metrics and port recognition.
• ⚠Resource-intensive operations: May exceed allocated limits if not properly configured.

Available in stages

Installation

Example stacks