X

Alejandro Vargas' Blog

  • May 1, 2019

The Power of HDFS ACLS

Alejandro Vargas
Technical Leader, ACS Global Delivery, Infrastructure & BigData

Kerberos and Sentry help us defining access privileges for a limited set of object living on HDFS

We can extend the control by using access control lists, ACL's 

Requesting the ACL's on a directory will show us if "dfs.namenode.acls.enabled" is set to tru on HDFS:

[root@linux-x64]# hdfs dfs -getfacl /user
# file: /user
# owner: hdfs
# group: supergroup

getfacl: The ACL operation has been rejected.  Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false.

In this case as it is set to false we need to enable it, for this we go to Cloudera Manager > HDFS > Configuration and search for "dfs.namenode.acls.enabled" and we mark it as enabled.

We need to restart the stale services to enable ACL's

First I'll create a new user for testing

[root@sf_D_DRIVE]# adduser avargas

[root@media]# grep avargas /etc/passwd
avargas:x:502:504::/home/avargas:/bin/bash
Then I will create a user hdfs a directory inside HDFS for user avargas
[root@media]# sudo su -hdfs
-bash-4.1$ hdfs dfs -mkdir /user/avargas
-bash-4.1$ hdfs dfs -ls /user
Found 9 items
drwxr-xr-x   - hdfs     supergroup          0 2019-04-29 12:42 /user/avargas
drwxr-xr-x   - cloudera cloudera            0 2019-01-24 08:45 /user/cloudera
drwxr-xr-x   - mapred   hadoop              0 2018-12-24 23:39 /user/history
drwxrwxrwx   - hive     supergroup          0 2017-10-23 09:17 /user/hive
drwxrwxrwx   - hue      supergroup          0 2018-12-19 07:44 /user/hue
drwxrwxrwx   - oozie    supergroup          0 2017-10-23 09:16 /user/oozie
drwxrwxrwx   - root     supergroup          0 2017-10-23 09:16 /user/root
drwxr-xr-x   - hdfs     supergroup          0 2017-10-23 09:17 /user/spark
As the owner user hdfs I'll assign rwx permissions on the directory /user/avargas to user avargas
sh-4.1$ hdfs dfs -setfacl -m user:avargas:rwx /user/avargas
-bash-4.1$ hdfs dfs -getfacl /user/avargas
# file: /user/avargas
# owner: hdfs
# group: supergroup
user::rwx
user:avargas:rwx
group::r-x
mask::rwx
other::r-x
Now I'll upload a file as user avargas
root@sf_D_DRIVE]# su - avargas
[avargas@~]$ hdfs dfs -ls /user > filelist.txt
[avargas@~]$ hdfs dfs -put filelist.txt /user/avargas
File /user/avargas/filelist is owned by user avargas:supergroup
[avargas@~]$ hdfs dfs -ls /user/avargas
Found 1 items
-rw-r--r--   1 avargas supergroup        680 2019-04-29 13:26 /user/avargas/filelist.txt
Directory /user/avargas still owned by hdfs:supergroup, but note the + sign indicating additional permissions assigned on it:
[avargas@~]$ hdfs dfs -ls /user | grep avargas
drwxrwxr-x+  - hdfs     supergroup          0 2019-04-29 13:26 /user/avargas

We can also set ACL’s for several users and groups on a single command

-bash-4.1$ hdfs dfs -setfacl -m user:avargas:rwx,user:hadoop:rwx,group::rwx,other::rwx /user/avargas

-bash-4.1$ hdfs dfs -getfacl /user/avargas
# file: /user/avargas
# owner: hdfs
# group: supergroup
user::rwx
user:avargas:rwx
user:hadoop:rwx
group::rwx
mask::rwx
other::rwx

Note that permissions were not changed for a file

-bash-4.1$ hdfs dfs -ls /user/avargas
Found 1 items
-rw-r--r--   1 avargas supergroup        680 2019-04-29 13:26 /user/avargas/filelist.txt

We can set ACL’s for several users and groups and recursively on all folders and files using the -R flag

-bash-4.1$ hdfs dfs -setfacl -m -R user:avargas:rwx,user:hadoop:rwx,group::rwx,other::rwx /user/avargas

-bash-4.1$ hdfs dfs -ls /user/avargas
Found 1 items
-rw-rwxrwx+  1 avargas supergroup        680 2019-04-29 13:26 /user/avargas/filelist.txt

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.