Amazon Web Services has added a managed migration capability to Amazon S3 Tables for converting eligible Apache Hive tables to Apache Iceberg. The API-driven process is intended to reduce the engineering and infrastructure work involved in modernizing data lakes while allowing source workloads to continue operating during most of the migration.
The service supports Parquet tables stored in Amazon S3 and registered in the AWS Glue Data Catalog. At launch, source and destination must be in the same AWS account and AWS Region. AWS says S3 Tables manages validation, compatibility checks, type conversions, file recompression and compaction as part of the migration workflow.
Migration process and operational considerations
Users select an S3 Tables destination table bucket, then create a namespace and table name for the new Iceberg table. AWS provides recommended default settings for query performance and storage efficiency. Before copying data, the service identifies the source objects in scope and produces a compatibility report, giving users information about whether the table can be migrated.
The destination remains read-only while the migration is in progress. Administrators can run synchronization operations repeatedly to bring it up to date with changes made to the source table; each sync copies only changes. This allows teams to defer cutover until they are ready. AWS says source writes do not need to stop during these syncs, although they are interrupted for a few minutes during cutover. Once migration is completed, the destination is promoted to read-write.
The migration can be initiated through the Amazon S3 console or programmatically using AWS CLI commands and AWS SDKs. AWS also identifies the Agent Toolkit for AWS as an option. Its migration skill for coding agents can analyze existing IAM policies and recommend corresponding destination policies, validate a migration plan, run the Start, Sync and Complete API calls, and assist with progress monitoring and validation.
Why move Hive tables to Iceberg
Apache Iceberg is an open table format for large analytics datasets. Its capabilities include schema evolution, hidden partitioning and time-travel queries. Because Iceberg tables can be accessed by compatible engines, adopting the format can make table data usable across a range of tools without tying it to one query engine.
S3 Tables provides managed storage designed for Iceberg tables, with automatic maintenance and optimization, replication, and S3 Intelligent-Tiering among its features. AWS says the migration charge is based on the amount of data copied, with no separate charge for converting Hive tables to Iceberg. Source objects already in S3 Intelligent-Tiering remain in their existing access tier during migration.
After cutover, users can work with the new tables through direct integrations with Amazon Athena and Amazon SageMaker Unified Studio in the AWS console. They can also query them using tools compatible with the Apache Iceberg REST API. These options give teams a path to use the migrated data without requiring the migration itself to dictate which compatible engine they use.
Availability and limits
AWS says managed migration is available in US East (N. Virginia and Ohio), US West (Oregon), Asia Pacific (Mumbai, Singapore, Sydney and Tokyo), and Europe (Frankfurt, Ireland and Stockholm). The initial scope is limited to Glue-registered Hive tables in Parquet format, with source and destination located in the same account and Region. Organizations with other source formats or cross-account and cross-Region requirements will need to account for those constraints when evaluating the feature.
The managed workflow addresses several tasks that AWS says could previously require weeks of engineering effort, including infrastructure management, data-type conversion and rollback planning. For teams whose tables meet the launch requirements, the staged process—compatibility review, copying, repeated synchronization and final completion—offers a way to move to Iceberg while limiting the period when the table is unavailable for writes.
