Monday, 30 September 2019

Now AWS Glue assists wheel files as dependencies for Glue Python Shell jobs

AWS Glue is an ETL service (extract, transform, and load) that is completely organized to make it easy to classify your data, polished it, enhance it, and transfer it accurately between different data stores. AWS Glue made up of a central metadata repository called as AWS Glue Data Catalog, an ETL engine that automatically creates Python or Scala code, and a flexible scheduler that handles dependency resolution, job monitoring, and retries. AWS Glue is serverless, so there’s no infrastructure to set up or manage. Earlier, you just allowed to add python dependencies with the help of egg files to AWS Glue Python Shell jobs. Now, with the help of wheel files, you can add python dependencies to AWS Glue Python Shell jobs, allowing you to get benefits of new potentials of the wheel packaging format. And now this feature is accessible in every AWS Region where AWS Glue is accessible. To get the complete list of AWS Region where AWS Glue is available, refer AWS Region page. To read more about this feature, read documentation.

No comments:

Post a Comment

AWS CodeGuru Elevating Code Security

  Security and code quality are paramount in today’s fast-paced software development landscape. As the cornerstone of DevSecOps, Static Appl...