Monday, 30 September 2019

Now AWS Glue assists wheel files as dependencies for Glue Python Shell jobs

AWS Glue is an ETL service (extract, transform, and load) that is completely organized to make it easy to classify your data, polished it, enhance it, and transfer it accurately between different data stores. AWS Glue made up of a central metadata repository called as AWS Glue Data Catalog, an ETL engine that automatically creates Python or Scala code, and a flexible scheduler that handles dependency resolution, job monitoring, and retries. AWS Glue is serverless, so there’s no infrastructure to set up or manage. Earlier, you just allowed to add python dependencies with the help of egg files to AWS Glue Python Shell jobs. Now, with the help of wheel files, you can add python dependencies to AWS Glue Python Shell jobs, allowing you to get benefits of new potentials of the wheel packaging format. And now this feature is accessible in every AWS Region where AWS Glue is accessible. To get the complete list of AWS Region where AWS Glue is available, refer AWS Region page. To read more about this feature, read documentation.

No comments:

Post a Comment

Kickstart Your GenAI Journey with Amazon Bedrock & Boto3 — How to Send Files, Handle Unsupported Formats & Start Exploring

✅ No S3 buckets ✅ No web frameworks ✅ Just plain Python, a file, and a smart AI agent ✅ BONUS: Learn how to handle Excel/Word files even if ...