Skip to content

dtrace: failed to grab pid ?!

December 17, 2012

There seems to be a reasonable security feature in Solaris that limits usage of diagnostics tools based on privileges(5) of source and target PIDs. Saying that, tools  such as dtrace(1M) and pstack(1) should have equal or more privileges(5) than the target PID they want to observe. Otherwise the process owner can use the target PID to run instrumented instructions with higher privileges which is obviously a security hole.

But this fair statement can cause some headache especially when processes start from sources other than a shell such as SMF.

Let’s examine this scenario. What prevents the owner of this process to look inside even after setting all dtrace permissions to zone and user.

$ ppriv $$
29763: -bash
flags = <none>
E: basic,dtrace_proc,dtrace_user,proc_owner
I: basic,dtrace_proc,dtrace_user,proc_owner
P: basic,dtrace_proc,dtrace_user,proc_owner
L: basic,contract_event,contract_observer,dtrace_proc,dtrace_user,file_chown,file_chown_self,
file_dac_execute,file_dac_read,file_dac_search,file_dac_write,file_owner,file_setid,ipc_dac_read,
ipc_dac_write,ipc_owner,net_bindmlp,net_icmpaccess,net_mac_aware,net_privaddr,net_rawaccess,
proc_audit,proc_chroot,proc_lock_memory,proc_owner,proc_setid,proc_taskid,sys_acct,sys_admin,
sys_audit,sys_mount,sys_nfs,sys_resource
$ /usr/ucb/ps auxwww | grep java | fgrep "XX:+ExtendedDTraceProbes" | awk '{print $1,$2}'
weblogic 1745
weblogic 27317
$ pfexec dtrace -n 'hotspot$target:::object-alloc{ @ = quantize(arg1) }' -p 1745
dtrace: failed to grab pid 1745: permission denied
$ pstack 1745
pstack: cannot examine 1745: permission denied

view raw
gistfile1.sh
hosted with ❤ by GitHub

What’s up? The reason is hiding somewhere in the SMF service manifest. Let’s have a look:

<exec_method name='start' type='method' exec=…>
<method_context>
<method_credential user='weblogic' privileges='basic,sys_resource,…'/>
</method_context>
</exec_method>

view raw
gistfile1.xml
hosted with ❤ by GitHub

Looking at the PID of the service we notice it has extra sys_resource privilege assigned via SMF that we don’t have in our bash PID ($$).

$ ppriv 1745
flags = <none>
E: basic,sys_resource,…
I: basic,sys_resource,…
P: basic,sys_resource,…
…..

view raw
gistfile1.sh
hosted with ❤ by GitHub

That’s preventing bash PID to access SMF started service PID although they are owned by the same user. So what is this extra privilege?

$ ppriv -lv sys_resource
sys_resource
Allows a process to modify the resource limits specified
by setrlimit(2) and setrctl(2) without restriction.
Allows a process to exceed the per-user maximum number of processes.
Allows a process to extend or create files on a filesystem that
has less than minfree space in reserve.

view raw
gistfile1.txt
hosted with ❤ by GitHub

That’s a necessary  for this service. So have to add the missing to the user:

usermod -K defaultpriv=basic,sys_resource,… weblogic

view raw
add_priv.sh
hosted with ❤ by GitHub

Next time bash has sys_resource privilege and executes dtrace/pstack against SMF service successfully:

$ ppriv $$
8630: -bash
flags = <none>
E: basic,dtrace_proc,dtrace_user,proc_owner,sys_resource
I: basic,dtrace_proc,dtrace_user,proc_owner,sys_resource
P: basic,dtrace_proc,dtrace_user,proc_owner,sys_resource
L: basic,contract_event,contract_observer,dtrace_proc,dtrace_user,file_chown,file_chown_self,file_dac_execute,file_dac_read,file_dac_search,file_dac_write,file_owner,file_setid,ipc_dac_read,ipc_dac_write,ipc_owner,net_bindmlp,net_icmpaccess,net_mac_aware,net_privaddr,net_rawaccess,proc_audit,proc_chroot,proc_lock_memory,proc_owner,proc_setid,proc_taskid,sys_acct,sys_admin,sys_audit,sys_mount,sys_nfs,sys_resource
$ pfexec dtrace -n 'hotspot$target:::object-alloc{ @ = quantize(arg1) }' -p 1745
dtrace: description 'hotspot$target:::object-alloc' matched 1 probe
^C
value ————- Distribution ————- count
536870912 | 0
1073741824 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 8945
2147483648 | 0

view raw
gistfile1.sh
hosted with ❤ by GitHub

From → dtrace, solaris

Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

<span>%d</span> bloggers like this: