linux - Process killed on ssh disconnect despite screen/nohup
2014-07
I am facing a rather strange situation:
- ssh to a beaglebone (details : uname -a = "Linux beaglebone 3.2.34 #1 Wed Nov 21 14:17:11 CET 2012 armv7l GNU/Linux", ssh server : Dropbear sshd v2012.55)
- launch any kind of process through screen, or nohup or /etc/init.d/
- logout
- re-ssh into it
- observe that the process is no longer there..
When using a second ssh connection, I can observe that the launched process is killed at the disconnection.
I've seen posts like What exactly determines if a backgrounded job is killed when the shell is exited, or killed?, but still can't understand this behaviour, which is clearly not the way screen
and other disowned processes are supposed to work.
$ shopt huponexit
huponexit off
I had to resort to using cron commands to persist the process
Why are detached processes killed at disconnection ?
Do you see other things to look for ?
It seems strange nohup does not work, but that can easily be put to test, as follows:
{ sleep 999; echo $? > exitcode ; } &
fuser -1 -k /bin/sleep
expr $(cat exitcode) - 128
This will print the return code minus 128, which is exactly the number of the signal that killed it. You can list them simply by doing:
kill -l
Now try this instead:
rm exitcode
{ nohup sleep 999; echo $? > exitcode; } &
fuser -1 -k /bin/sleep
ls -l exitcode
If nohup works, at this point you will be told that there is no such file. You can double-check this by doing:
fuser -15 -k /bin/sleep
expr $(cat exitcode) -128
and finding this value to be 15.
EDIT
Your last comment was quite revealing: it means the SIGHUP is not sent to the process (sleep, in this case) but directly to your shell. This can be done ony by Dropberar, of course. A little research showed that Dropbear does indeed kill all user processes at logoff.
You can turn this annoying feature by adding the line
KillMode=process
at the end of the Service stanza in /lib/systemd/[email protected] file. Then either reboot or restart Dropbear.
I run
nohup bash -c "while [ true ]; do echo test; done"
from PuTTy
SSH client, but after this process is not running in background instead nohup
keeps to be foreground process in shell.
SW version: nohup (GNU coreutils) 8.5
nohup bash -c "while [ true ]; do echo test; done" &
Nohup provides you immunity to hangup signals. But it does not automatically set the process to background.
It's the & at the end of the command that causes it to run in the background.