AWS re:Invent Updates: Nov 28 - Dec 2

Keeping up with the flood of AWS re:Invent announcements can be a challenge. This article summarizes and highlights the key pre-announcements to save you time and keep you informed about the latest developments in data systems.

---

Databases

Fleet-level Database Insights in Amazon Aurora PostgreSQL and MySQL

As organizations grow they commonly end up running dozens or even hundreds of database instances. AWS now offers a slew of new capabilities to make it easier to understand what is happening across the fleet. Read the blog.

---

Generative AI in Database Migrations

AWS Data Migration Service is designed to make it faster and easier to translate applications designed for proprietary databases to work with open source databases like PostgreSQL. This is a natural fit because generative AI has a strong track record both in translating human languages and manipulating software code. Read the blog.

---

Oracle Exadata Database on AWS

Oracle’s Exadata database platform handles some of the demanding workloads. Until recently, only Oracle’s cloud had the specialized hardware needed for running Exadata, but it is now becoming available on other cloud platforms, including now on AWS in limited preview. Read the press release.

---

Multi-Region Support in MemoryDB

Amazon MemoryDB provides a robust implementation of the fast and efficient Redis protocol. Within one region, it already runs as a distributed system, using Conflict-free Replicated Data Types (CRDTs) to reconcile conflicting concurrent writes. Now multi-region support is generally available, providing active-active replication the same microsecond read, and single-digit millisecond writes on a geographically replicated data set. Read the blog.

---

File Systems and Object Storage

Feed GPUs at 1.2 Terabits per Second—12x Faster

Lustre is a file system originally developed for supercomputing that has found a new range of applications in AI. GPUs are data-thirsty and this release combines the the AWS Elastic Fiber Adapter (EFS) with NVIDIA GPUDirect Storage (GDS) to provide a highly path between storage and GPUs. Read the blog.

---

End-to-End Checksums to Protect S3 Data

S3 object storage has always had high standards for data integrity, but a gap in protection existed because the checksums S3 used internally to verify data were computed separately from those used during data transfer. Client-generated data checksums help preserve data integrity along the entire path from storage to retrieval. S3 has had this capability for two years and now turned it on by default. Read the blog.

---

No-Code Web Apps for S3 Transfers

AWS Transfer Family now lets you create fully managed web sites that allow authorized users to upload and download files from S3 storage. This means you can quickly spin up secure portals allowing non-technical personnel to interact with cloud resources and applications. Read the blog.

---

AWS Physical Data Transfer Terminals

AWS now provides locations in Los Angeles and New York where you can bring your storage devices to gain access to fast upload speeds, directly to the AWS cloud network. Read the blog.

---

Automatic Storage-Class Tiering for Network File Systems

Access patterns and performance requirements often varies across files and over time. For example, recent files might be accessed more frequently than older files, and some logs files might only rarely be read. AWS FSx for OpenZFS now intelligently shuffles data between different SSD and HDD storage to provide a balance of cost and performance. Read the blog.

---

Target Completion Times for EBS Snapshots

EBS snapshots provide point-in-time images of a block device. It can take a long time to complete the snapshot process because it treated as a background job—rate limited so it does not impact production. This is problematic for things like backups, which need to complete within a certain time window, so EBS now allows you to set a snapshot completion time and, if necessary, provision additional throughput to meet it. Read the blog.

---

Stay tuned for more updates as AWS re:Invent continues!