Implement cloud-based backups for Plastic Cloud
I propose that Plastic Cloud offers an incremental backup method. It should export backup data to customer-provided cloud storage (AWS S3 / Google Cloud Storage / Azure Storage), perform backups with high frequency (possibly with a combination of full / incremental backups), and only leave the responsibility of validating these backups with the customer.
Our use case:
We use Plastic Cloud instead of Plastic Server, because it reduces our operational overhead. Plastic Cloud is great in many ways, but using it means that we also entrust the responsibility of availability, security, as well as backups/disaster recovery with Codice Software.
As our organization grows larger and older, we need to have shared responsibility for backups/disaster recovery. If we ask ourselves, "do we have backups?" then we must be able to answer "yes, and we have verified that they are not corrupt", not "yes, we trust that another organization does it for us".
It is possible to handle backups today: run a local Plastic Server, scan the Plastic Cloud organization, replicate what's in there to local repositories, then archive the local repositories. However, that requires a fair bit of operational work.
I would like to see that Plastic Cloud offered an 'automated backups' feature: I provide details on a customer-controlled cloud storage bucket in a cloud provider like AWS/GCE/Azure, and credentials that allow Plastic Cloud write-only rights to the bucket (without the rights to modify what's already in the bucket). Plastic Cloud would then regularly export changes to the bucket. For example, a full export once per month + an incremental export once per day. I would like notifications to be sent to me (for example via emails) whenever there is a problem with export process, as well as summaries of this somewhere (for example within the weekly digest emails that are sent out).
I would like validation to be left with the customer (although someone will probably ask for some degree of automation here as well). As such, I expect that I can restore a backup - or perhaps part of a backup, like, only a single repo - into a local Plastic Server and then be able to access the contents directly.
The automated "export to cloud storage bucket" process will allow us to easily implement any process we would like for further backups (for example, manually copy backups from cloud bucket to offline storage).
It would be helpful to be able to be able to get some idea of the contents of a backup without needing to perform a full restore. (Example: which repos are inside the backup? what is the range of changesets contained within the backup?) This could either be a tool that inspects the backup files itself, or additional text file(s) that are written at the time when the backups are created.
One nice side effect of a backup mechanism like this is that it provides us with the ability to recover from some of our mistakes, that would otherwise require contacting Codice support.
The feature requires additional backend-side processing and network transfer. We are happy to pay extra for this.
(Slightly related, there is a cloud-to-cloud backup service for GitHub: https://backhub.co/ - this has a much broader featureset than suggested above, but it demonstrates the core need.)
-
Albert J. commented
Will also add Wasabi as a provider (https://wasabi.com/)
-
Hi Mikael,
This is definitely an awesome feature. I'd love to start working on it asap. But, it is not a trivial one. There are many steps involved.
We are about to do a big redesign of our current Cloud, and it will make it much closer to the current Jet backend, so this sort of stuff will be much easier to do.
We'll keep an eye on it. Meanwhile, let's see if more people is interested.
Thanks!
pablo