Last updated
Learn & practice AWS Hacking: Learn & practice GCP Hacking:
AWS's Elastic MapReduce (EMR) service, starting from version 4.8.0, introduced a security configuration feature that enhances data protection by allowing users to specify encryption settings for data at rest and in transit within EMR clusters, which are scalable groups of EC2 instances designed to process big data frameworks like Apache Hadoop and Spark.
Key characteristics include:
Cluster Encryption Default: By default, data at rest within a cluster is not encrypted. However, enabling encryption provides access to several features:
Linux Unified Key Setup: Encrypts EBS cluster volumes. Users can opt for AWS Key Management Service (KMS) or a custom key provider.
Open-Source HDFS Encryption: Offers two encryption options for Hadoop:
Secure Hadoop RPC (Remote Procedure Call), set to privacy, leveraging the Simple Authentication Security Layer.
HDFS Block transfer encryption, set to true, utilizes the AES-256 algorithm.
Encryption in Transit: Focuses on securing data during transfer. Options include:
Open Source Transport Layer Security (TLS): Encryption can be enabled by choosing a certificate provider:
PEM: Requires manual creation and bundling of PEM certificates into a zip file, referenced from an S3 bucket.
Custom: Involves adding a custom Java class as a certificate provider that supplies encryption artifacts.
Once a TLS certificate provider is integrated into the security configuration, the following application-specific encryption features can be activated, varying based on the EMR version:
Hadoop:
Might reduce encrypted shuffle using TLS.
Secure Hadoop RPC with Simple Authentication Security Layer and HDFS Block Transfer with AES-256 are activated with at-rest encryption.
Presto (EMR version 5.6.0+):
Internal communication between Presto nodes is secured using SSL and TLS.
Tez Shuffle Handler:
Utilizes TLS for encryption.
Spark:
Employs TLS for the Akka protocol.
Uses Simple Authentication Security Layer and 3DES for Block Transfer Service.
External shuffle service is secured with the Simple Authentication Security Layer.
These features collectively enhance the security posture of EMR clusters, especially concerning data protection during storage and transmission phases.
Learn & practice AWS Hacking: Learn & practice GCP Hacking:
Check the !
Join the 💬 or the or follow us on Twitter 🐦 .
Share hacking tricks by submitting PRs to the and github repos.
