Components
The installation of Data Analytics System's specific components, as well as all components it requires, is customized each time to incorporate new versions and the peculiar deployment methods that take into account security updates, bug fixes, and the introduction of new features.
In addition to Data Analytics System's specific components, its operation also requires the following components.
KeyCloak
This is the component used to manage authentication and authorization and, in addition to Single Sign-On (SSO) features, already integrates management of a second authentication factor (2FA).
MinIO
This is the component used as distributed storage compatible with S3 within the cluster and is the one that can most influence the sizing of the nodes' disks. Even if Longhorn is used as the default storage class, for MinIO disks, the local-storage class (local-path class in K3S) is recommended, using directly the local disks (i.e., not managed by other storage managers such as, for example, Longhorn).
The single "Drive" associated with each "Server" of Minio that runs on each core worker node is allocated on the local disk of the node itself and this must be taken into account when sizing such a disk. The size of this Drive also contributes to the overall size of storage seen by MinIO, which can be calculated using the Erasure Code (EC) calculator of MinIO (https://min.io/product/erasure-code-calculator).
For example, if each drive has a capacity of 8 TB and 4 Servers are available, each with a single drive, accepting an EC equal to 2, then the usable net capacity is 16 TB (with a fault tolerance of 2 nodes).
Redis
This is the component used as an in-memory database for caching.
Kafka
This is the distributed messaging system.
Argo Workflow
This is the component used as a workflow orchestrator.
ArgoCD
This is the component used as a tool for deploying applications according to GitOps methods.