HPC Parallel File Systems across clouds
Parallel file systems for HPC and training - FSx for Lustre, Azure Managed Lustre, Parallelstore - fronting an object store, with the durable copy in the bucket and the fast copy sized on the step.
3 verified modules - all static-validated and publish-checked; live-test pending.
Compare by provider
| Provider | Module | Verification |
|---|---|---|
| AWS | FSx for Lustre, Persistent by Default | static-validated |
| Azure | Managed Lustre with Blob Integration, Root Squash and Capacity on the SKU's Step | static-validated |
| Google Cloud | A Persistent Parallelstore Instance on the Private Services Range You Already Have | static-validated |
How to choose
Decide first what the durable copy is, because none of these is it: FSx and Managed Lustre link a bucket or a container, and Parallelstore imports from and exports to Cloud Storage. Then compare the throughput tier and its capacity step, whether root on a client is squashed, whether the file system is zonal, and what the minimum size costs per hour.
When not to use
These are scratch tiers with a persistence option, and the option is not a backup. A file system deleted with data that was never exported is data gone; the bucket is the record and the file system is the cache in front of it.