AWS S3 (Simple Storage Service) plays key role in modern data architecture as a centralized enterprise data lake layer for all the stages from ingestion, transformation, visualization, backup and DR, archival solutions. As a data lake, AWS S3 provides a centralized storage layer capable of storing structured, semi-structured, and unstructured data from diverse sources including:

• Relational databases (RDBMS)
• APIs
• IoT devices
• External data sources

Integration with Internal Business Units:
Many organizations have internal business units that manage their own datasets manually. These datasets often need to be integrated with centralized data lakes and data warehouses hosted in the cloud.
To support this:

• Enforce secure data practices, read and write access should be granted only to specific S3 prefixes based on user roles and responsibilities. This ensures that end users can access only the data relevant to their business unit or function.
• Access permissions must be configured in AWS S3 to allow end users to upload their datasets into designated S3 buckets.
• Since these datasets are often restricted to specific business units, access controls must be enforced accordingly.

Access Control Considerations:
While row-level access controls are commonly implemented in data warehouses, similar access restrictions should also be applied at the S3 data lake level. This ensures:

• Only authorized users can access specific datasets.
• Business unit-level data segregation is maintained.

Example Scenario:
Let’s understand this with a practical example (to be detailed next), illustrating how access permissions and data integration work in a real-world AWS S3 data lake setup.

Business Unit Access Control in AWS S3 Data Lake:
In a typical enterprise setup, business units such as Risk, Cards, and Loans should have read and write access only to their respective S3 prefixes. This ensures that each unit can access and manage only the datasets relevant to them.

IAM Role-Based Access Design:
To enforce this segregation:

• Separate IAM roles should be created for each business unit.

• End users from each business unit should be mapped to their respective IAM role.

Example: Risk Business Unit:
For instance, the IAM role risk-data-lake can be configured to allow access only to the S3 prefix:
s3:///pba/manual/raw/risk

This role would grant:

• Read and write permissions

• Access restricted to the Risk unit’s data only

This approach ensures data security, compliance, and operational clarity across business units.
It can be implemented with below policy created and role is created using it and users are mapped to this role.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Action": "s3:ListBucket",
            "Effect": "Allow",
            "Resource": "arn:aws:s3:::mycompany-dev-landing",
            "Condition": {
                "StringEquals": {
                    "s3:prefix": [
                        "",
                        "pba/",
                        "pba/manual/",
                        "pba/manual/raw/",
                        "pba/manual/raw/risk/"
                    ],
                    "s3:delimiter": [
                        "/"
                    ]
                }
            }
        },
        {
            "Sid": "VisualEditor0A",
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::mycompany-dev-landing",
            "Condition": {
                "StringLike": {
                    "s3:prefix": "pba/manual/raw/risk/*"
                }
            }
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject"
            ],
            "Resource": "arn:aws:s3:::mycompany-dev-landing/pba/manual/raw/risk/*"
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": "s3:GetBucketLocation",
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Sid": "VisualEditor3",
            "Effect": "Allow",
            "Action": "s3:ListAllMyBuckets",
            "Resource": "arn:aws:s3:::*"
        }
    ]
}

This policy allows below permissions:

• List all the buckets in AWS S3

• List the all the objects in mycompany-dev-landing bucket

• Read and write objects to specific prefix - pba/manual/raw/risk/*

Conclusion:
This approach enables the implementation of restricted access in the AWS S3 data lake, aligning with business unit boundaries and compliance requirements.