ChartMuseum Replication support for Harbor >2.8

[Vad1mo]

Starting with release 2.8 Harbor is removing ChartMuseum.
While Harbor now only supports OCI artifacts, the majority of Helm Charts are in ChartMuseum format.
Helm is not deprecating or replacing the ChartMuseum Format with OCI. Both storage option will coexist in Helm.

Describe the solution you'd like
While Harbor does not support ChartMuseum any more it would be a practical option to support the replication of Helm Charts from ChartMuseum and convert them to OCI. One Way. Mostly third-party and external registries.

Describe the main design/architecture of your solution

• New Registry Endpoint in Harbor to being able to select a ChartMuseum endpoint.
• One way replication rule to replicate ChartMuseum to Harbor OCI.

Describe the development plan you've considered
Converting a Chart to OCI is technically simple, so there is no need to reimplement ChartMuseum.
Harbor needs to be able to parse the index.yaml file and download the Charts and push it to OCI registry.

Additional context
Ongoing Discussion
Community Meeting discussion on January 11, 2023 link to recording

[wy65701436]

The original idea of replication was to copy docker images from registry to registry (push or pull) and then extend to OCI compatible files. From this perspective, this is clearly not true.

I disagree with some other angles of this idea.

1. In order to transfer chart museum files to OCI files, a third party tool or binary needs to be introduced in Harbor-core, which brings a lot of drawbacks, such as upgrade maintenance, CVE management. And, what if users do not want to use the tools we provide?
2. Replication rules are not compatible with chartmuseum files, such as push mode and chunked replication. We need special treatment.

My main reason is why we break the interface design for an obsolete supported file format, instead of providing a tool to help users convert?

[chlins]

We need to maintain the API caller code in the adapter for the chartmuseum if we want to support replication, in my opinion, it's not a good practice to keep and maintain the code which related to the thing has been deprecated.

[stonezdj]

It is better to provide a third-party tool to replicate/migrate charts from chartmuseum oci registry.

[MShekow]

It is better to provide a third-party tool to replicate/migrate charts from chartmuseum oci registry.

We have made good experiences with https://github.com/bitnami-labs/charts-syncer although there is still an issue that blocks is from using it.

[Vad1mo]

• In order to transfer chart museum files to OCI files, a third party tool or binary needs to be introduced in Harbor-core, which brings a lot of drawbacks, such as upgrade maintenance, CVE management. And, what if users do not want to use the tools we provide?

This is no needed, there are only a few API calls needed that can be done straight in Harbor

• Replication rules are not compatible with chartmuseum files, such as push mode and chunked replication. We need special treatment.

The API and UI interfaces need to be extended, to support that.

[SOMAPANGU]

so, Is there any better way to replicate ChartMuseum to Harbor OCI?

[github-actions]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[MShekow]

From what I can tell, using https://github.com/bitnami-labs/charts-syncer (e.g. in a cron job or scheduled CI pipeline) is really the best option. The issue I mentioned above is really not an issue in practice: you only need to make sure to use the --skip-dependencies flag when invoking charts-syncer. Any packaged Helm chart is self-sufficient anyway, meaning that a tar.gz file already contains any potential sub-charts, so there is no need to have charts-syncer also explicitly sync those (which is its default behavior, but it causes crashes for Helm charts that reference charts stored in other registries than the main chart).

[github-actions]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[github-actions]

This issue was closed because it has been stalled for 30 days with no activity. If this issue is still relevant, please re-open a new issue.