> ## Documentation Index
> Fetch the complete documentation index at: https://docs.costgraph.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Changelogs
Releases for the CostGraph operator will have helm tags associated that are not tied to the release version. Across helm versions within a major or minor release, patch updates will be made for hotfixes, bugs and business focused updates. Helm release notes are available on our [chart repository](https://github.com/BaselineHQ/costgraph-charts), view the helm [index.yaml](http://charts.costgraph.ai/index.yaml) to find information about the latest releases.
## SLA
* We will provide changelogs for major and minor releases of the operator
* Customers will be made aware of any breaking changes in any release cycle
* We aim to provide releases on a regular basis for customer features and business priorities
* All related tag updates will be made under a helm release, no update is needed from customers for operator dependencies
Entries labelled `costgraph-operator vX.Y.Z` are releases of the current `costgraph-operator` chart, which deploys the domain-specific operators. Earlier entries labelled `vX.Y.Z` are releases of the deprecated single-binary `costgraph` chart. The two version series are independent, so the numbers do not run continuously.
### Breaking Changes
* **`helm upgrade` fails until the two operator Deployments are deleted.** `costgraph-operator-kubernetes` and `costgraph-operator-prometheus` are now rendered by a shared chart, and their `spec.selector` changes as a result. A Deployment's selector is immutable, so Kubernetes rejects the patch rather than replacing the object. Delete both before upgrading; they are stateless and are recreated immediately:
```sh theme={null}
kubectl -n delete deployment \
-costgraph-operator-kubernetes \
-costgraph-operator-prometheus
helm upgrade costgraph/costgraph-operator
```
No metrics are lost. The operators resume from the cluster's current state on restart, and the collectors that hold data are DaemonSets, which are unaffected.
* **Values under `operatorKubernetes` and `operatorPrometheus` move beneath `deployment`.** A values file that sets them at the old paths still parses, but the settings no longer apply and the chart's defaults are used instead. Nothing errors, so check any override you rely on:
| Before | After |
| ------------------------------------ | -------------------------------------------------------- |
| `operatorKubernetes.image.tag` | `operatorKubernetes.deployment.image.tag` |
| `operatorKubernetes.replicas` | `operatorKubernetes.deployment.replicas` |
| `operatorKubernetes.resources` | `operatorKubernetes.deployment.resources` |
| `operatorKubernetes.enabled` | `operatorKubernetes.deployment.enabled` |
| `operatorKubernetes.nodeSelector` | `operatorKubernetes.deployment.nodeSelector` |
| `operatorKubernetes.tolerations` | `operatorKubernetes.deployment.tolerations` |
| `operatorKubernetes.affinity` | `operatorKubernetes.deployment.affinity` |
| `operatorKubernetes.podAnnotations` | `operatorKubernetes.deployment.additionalPodAnnotations` |
| `operatorKubernetes.podLabels` | `operatorKubernetes.deployment.podLabels` |
| `operatorKubernetes.securityContext` | `operatorKubernetes.deployment.securityContext` |
The same mapping applies to `operatorPrometheus`. Settings under `config`, and the top-level `global` and `rbac` values, are unchanged.
### Updates
* The two operator Deployments are rendered from a shared definition rather than a template each, so they no longer drift apart in the fields that should match
* The flowtrace DaemonSet, the ClusterRole and its bindings, the ServiceAccount, the credentials Secret and every bundled exporter are unchanged
### Breaking Changes
* The operator's ClusterRole no longer grants `get`/`watch`/`list` on every resource in every API group. Built-in resources are granted by name and custom resources per API group. Secrets, ConfigMaps and ServiceAccounts are not part of the grant at any version, so the API server refuses them outright rather than the operator choosing not to read them
* Resources in the new `rbac.excludeAPIGroups` list are no longer collected. These are API groups carrying credential material or references to it (cert-manager, External Secrets, Velero, Traefik, Flux notification providers, certificates and the authentication/authorization/RBAC groups), plus cluster machinery with no cost or topology signal (events, leases, admission and API registration, flow control, metrics, scheduling). Dashboards referencing any of these will stop updating after upgrade
* Installing now requires a renderer with cluster access, because the set of custom resource API groups is discovered from the cluster. `helm install`, `helm upgrade` and Argo CD v2.10 or newer qualify. A render without it fails with an explanatory error rather than installing an operator that silently collects nothing beyond core Kubernetes. Set `rbac.requireClusterDiscovery=false` for a cluster that genuinely runs no custom resources
### Updates
* Workloads, nodes, namespaces, storage, networking and pod network identity are unaffected. Karpenter, Argo CD, Cilium, CNPG, cloud provider CRDs and every other custom resource the cluster serves continue to be collected without configuration
* New `rbac.excludeAPIGroups` value sets which API groups CostGraph may not read, applying to every version of a group rather than the specific versions named
* New `rbac.grantCustomResources` value collects built-in resources only when set to `false`
* New `rbac.extraRules` value appends rules to the ClusterRole verbatim. Use it to grant named resources from an excluded group, for example collecting Traefik `IngressRoute` topology while `Middleware`, which can hold inline credentials, stays unreadable
* The bundled kube-state-metrics no longer requests Secrets, ConfigMaps, certificate signing requests, leases or webhook configurations. Its upstream defaults granted read access to every Secret in the cluster to publish metadata CostGraph does not use
* Resources the operator is not permitted to read are skipped at startup rather than retried, so a narrower grant produces no errors and no restart loop
### Breaking Changes
* N/A
### Updates
* Added a DaemonSet-based, root-privileged BPF network-flow tracing agent that runs on compatible nodes across the fleet. Compatible nodes must expose a host-mounted cgroup v2 filesystem and run a BTF-enabled kernel for the fentry programs.
* Added default host filesystem mounts for `/proc:/host/proc` and the host cgroup v2 filesystem `/sys/fs/cgroup:/host/sys/fs/cgroup`
### Breaking Changes
* The `costgraph` chart is deprecated and no longer supported. It receives no further releases and no fixes. Install the `costgraph-operator` chart - see [Overview](/costgraph/operator/overview)
### Updates
* Both operators now ship as tagged releases, `v0.1.0`, and the chart pins those tags rather than build identifiers. You can now see exactly which operator version a chart release deploys via the chart `appVersion`
* Cluster inventory removes resources when they are deleted, so decommissioned workloads no longer linger in the dashboard
* Lower memory use when scraping high-cardinality exporters
* Default resource requests tuned to match observed usage of the operator components
### Breaking Changes
* N/A
### Updates
* Both operators expose profiling on the standard port `6060` for cluster-internal diagnostics, making support investigations faster
* Metric names and external labels are consistently namespaced under the `costgraph_` prefix across every exported series
### Breaking Changes
* N/A
### Updates
* The kubelet is scraped directly as a metrics destination, improving host and container metric coverage on clusters where cAdvisor is unavailable
* Additional diagnostics to help troubleshoot partially reporting clusters
### Breaking Changes
* N/A
### Updates
* Health markers are published for scraped resources, so clusters that are only partially reporting are now visible rather than silently incomplete
* Node scheduling keys are configurable for the operator components and their exporters
* Chart documentation refreshed for the split components
This release reaches feature parity with the `costgraph` chart, which is superseded from this point on.
### Breaking Changes
* N/A
### Updates
* Resource requests and limits are configurable for every bundled dependency, so the operator can be sized to the cluster it runs in
* Includes chart patch releases v0.1.8 and v0.1.9: node-exporter is bundled as a metrics source, scrape targets are discovered dynamically instead of being statically configured, and a Helm value override failure under ArgoCD is fixed
### Breaking Changes
* N/A
### Updates
* Cluster role rules corrected so that subresources are no longer requested with a wildcard, which some admission controllers rejected
* Includes chart patch releases v0.1.4 through v0.1.6: the API key can be supplied from an existing Secret, the backend URL is overridable for private deployments, and a rendering error caused by whitespace in release names is fixed
### Breaking Changes
* N/A
### Updates
* **Beta release of the Kubernetes operator.** The `costgraph-operator` chart now deploys the full split - `costgraph-operator-kubernetes` for cluster inventory and `costgraph-operator-prometheus` for metrics
* Resource discovery is restricted to API resources that actually support `watch` and `list`, removing repeated permission errors on clusters with custom aggregated APIs
* GPU metrics and rightsizing recommendations for GPU-backed nodes
With this release the `costgraph` chart is superseded. New installations should use the `costgraph-operator` chart.
### Breaking Changes
* N/A
### Updates
* A customer-supplied cAdvisor deployment can be used instead of the bundled one
* Scheduled reconciliation replaced with event-driven collection, so cluster changes are reflected without waiting for the next cycle
* Includes chart patch release v0.1.1: the bundled cAdvisor image maps PVC device information, which is required for volume cost attribution
### Breaking Changes
* N/A
### Updates
* **First release of the redesigned operator.** The single-binary operator is being broken out into components with a single responsibility each, released and versioned independently. This release ships the metrics component, which scrapes cluster, host and container metrics and remote-writes them to CostGraph
* Cluster identity is attached to every published metric, so multi-cluster accounts attribute data correctly
This chart is installed as `costgraph-operator` and is separate from the `costgraph` chart. Do not run both against the same cluster.
### Breaking Changes
* N/A
### Updates
* Disk and Network metrics are now available via the Gateway APIs, dashboards updates to support this feature are coming soon
* We now ignore right-sizings if the resource does not have valid settings to apply the recommendations.
* Prometheus library updated to have improved performance and stability
* Bugfixes and cleanup logic improvements
### Breaking Changes
* N/A
### Updates
* Prometheus interval bug causing CPU usage to be reported incorrectly is fixed, this is now autodiscovered from the prometheus configuration
* Inactive resources are now included in historical data to support rightsizing recommendations
* Provider metadata is now included in the exported metrics
* Annotations and labels are now included in the exported metrics
* Bugfixes and cleanup logic improvements
### Breaking Changes
* We will be removing the automatic deployment of superset and postgres in this release and for the future. With our now updated dashboard providing support for visuals, customers who want superset can now enable them on installation
If you have superset enabled, the instance will keep running. However, you will need to manually enable postgres and configure the operator to push data entries to continue having support for these instances
In the future, we will be removing the superset and postgres dependencies from the operator and they will be only supported by customer installations outside our helm charts
### Updates
* Support for GCP managed prometheus added to our environment, Workload Identity Federation is required for this feature. See [here](https://cloud.google.com/stackdriver/docs/managed-prometheus/setup-otel#gmp-prereqs) for more information
* Automated rightsizings are now available and can be enabled on a workload-by-workload basis.
* Idle usage is now part of the recommendation data, usage can be audited from the dashboard
* Performance improvements to the owner resource grouping process
* Cluster provider specific metadata is now included in the exported data
* Reconcile time period is now configurable.
### Breaking Changes
* N/A
### Updates
* Retry logic improvements for the prometheus base to ease large cluster throttling failures
* Consume labels and annotations as part of improvements to topology for cluster resources
* Instant usage metric is now available to view data about container resource dynamics
* Deploy resource filter to disambiguate emptyDir vs memory usage data
* Bugfixes and cleanup logic improvements
### Breaking Changes
* N/A
### Updates
* Cost reporting improvements to correct errors in usage metrics
* Bugfix correcting misrepresented memory values with emptyDir configurations
* Addition of backoff and retries to our codebase
* Performance and memory improvements dropping memory usage requirements by 50%
* `metric_window_days` parameter allowing custom intervals to perform recommendations.
### Breaking Changes
* N/A
### Updates
* Base memory usage improvements in the operator data aggregation pipeline
### Breaking Changes
* N/A
### Updates
* Upgrade of database provider module to the latest version
* Addition of Computed usage metrics to the data model
* Performance improvements and logging updates
### Breaking Changes
* Due to the deployment of our new pricing infrastructure, customers using the raw pricing data will see schema changes. As much as we attempt to provide a standard schema, this change was unavoidable. This change is limited to customers developing with the data from our operator, no changes will be observed for dashboard and prometheus experiences.
### Updates
* Deployment of a new pricing infrastructure bringing improved operator performance with lower latency
* Updates to the savings dashboard to show more than 5 entries previously causing customers to miss relatively smaller container resources
* Improved pricing updates supporting newer cloud instance types and various usage types such as spot and reserved
* Addition of per-CPU and per-Memory pricing for all cloud providers improving the Cost Driver dashboard experience
* Support for newer GCP instance types and their upcoming Sweden region release
* Addition of liveness and readiness probe endpoints for customer visibility on failed instance deployments
* Hotfix for GCP custom instance types deviating from standard pricing obtained from cloud providers
* Hotfix for AZ based AWS spot pricing on Opt-In regions
* Support for all regional DigitalOcean pricing information providing customers more value for this provider
* Future support for resource pricing updates in Storage, Network, GPUs and related resource types
### Breaking Changes
* N/A
### Updates
* Initial release
* Support for all major kubernetes cloud providers such as AWS, GCP, Azure
* Dashboards iterations with customers for their internal use cases
* API key support for multiple clusters within a single account
* RBAC and invitation support for organizations
* Clusters can now be viewed on the web dashboard