AWS Demonstrates Cross-Region AI Training With SageMaker HyperPod and Qumulo

AWS’s September 25, 2026 post describes pairing Amazon SageMaker HyperPod with Cloud Native Qumulo and Qumulo Cloud Data Fabric so training compute can run in one AWS Region while the dataset remains in another. NeuralCache predicts and caches data reads for the remote cluster, which accesses the dataset through a local mount. AWS says the approach avoids copying the full dataset for this configuration.

AWS ran the same 1.02-billion-parameter LLaMA v3 training job independently on two clusters: a data-co-located hub in US East (Ohio) and a spoke in US West (Oregon) reading data across Regions. Each cluster used two ml.p5.48xlarge instances, for 16 H100 GPUs in total. The post reports 60 milliseconds of network latency in the cross-Region test.

According to AWS, after NeuralCache warmed up, the remote cluster reached 115–117 samples per second, matching the co-located cluster; throughput converged within the first 100–150 batches. AWS also says the one-time warmup represented less than 1% of wall-clock time at 10,000 or more batches and under 0.1% at more than 100,000 batches. Those figures are AWS’s results for the specific configuration it tested.

The example offers one design for teams whose datasets and GPU capacity sit in different AWS Regions: use cache warmup to address cross-Region reads rather than assuming the full dataset must first be moved to the compute Region. AWS also published a separate September 29 tutorial on SkyRL post-training of Qwen3-VL-8B on HyperPod. That is a different workload and evaluation, not a directly comparable cross-Region throughput test.