ssh - Disabled root login and no root users.. What to do? CentOs 6

We have a couple of servers which are managed by a large group of admins. They usually log in as service users (say hudson) and then switch to root to make some small fix. This means we often can't map a change made to a person.

Does anyone have a script for Unix/Linux which can tell me which user logged in as root? Logins can be from all computers on the local LAN. Remote access from outside the LAN as root is not possible; admins must first login with a LAN user and can then promote themselves to root (they all use SSH).

What I would like is a script which follows the remote logins (in the local LAN) and print the user name for a certain time. You can assume that the script can login via ssh to any computer on the local LAN as root without being asked for a password.

Background: I have a script which saves backup copies of all files edited by root. The problem is to find out who really made the change.

Security is not an issue; this is not to find hackers which might have cleaned wtmp, it's to find out who made a mistake to give feedback.

[EDIT] Some pointers: The command last helps:

> last -t 20101029174200 root
root     pts/26       :0.0             Wed Oct 20 15:36 - 15:03  (23:27)    

wtmp begins Fri Oct  1 16:34:36 2010

So root was logged in via pts/26. Who else sat on that pseudo TTY?

> last -t 20101029174200 pts/26
adigulla pts/26       :0               Mon Oct 25 09:45   still logged in   
adigulla pts/26       :0               Fri Oct 22 14:00 - 17:29  (03:29)    
adigulla pts/26       :0               Thu Oct 21 15:04 - 16:05  (01:01)    
root     pts/26       :0.0             Wed Oct 20 15:36 - 15:03  (23:27)    
adigulla pts/26       :0.0             Fri Oct 15 15:57 - 15:57  (00:00)    

wtmp begins Fri Oct  1 16:34:36 2010

Hmm... must be me. So I can follow user changes on the local machine. If I log in to a remote machine:

$ last -1 hudson 
hudson   pts/0        192.168.0.51     Fri Oct 29 17:52   still logged in   

So I get the PTY and the IP address where I came from. How can I make the connection from the output of last for hudson to the user on 192.168.0.51?

[EDIT2] Please also note that we usually change user with ssh, not sudo or su. This allows for single sign on and avoids having to tell admins any passwords. If we want to grant/revoke access to something, we simply add/remove the public key from the service account. I also know that ssh logs to syslog but the messages don't tell me which user switched to root:

sshd[7460]: Accepted publickey for root from ::1 port 36689 ssh2

I think the best way to give feedback to admins who mistakenly logged in as root is to disable root logins whilst still allowing su and sudo. Alternatively have root's .profile display a suitable feedback message and optionally then exit.

If anyone other than an admin is mistakenly logging in as root, you definitely need to at least change root's password :-)

OK, you have revised the question to clarify what it is you want to achieve. You need to minimise the number of services that allow logins and ensure all of them records a successful login. SSH records successful logins via syslog.

If you have several people logging in as "hudson", you need to stop that. It should be a site policy that admins each log in using a unique userid.

You need to ensure that logfile rotation is adjusted so that you can search older logs for whatever period you fell necessary.

You could certainly have a script scheduled by cron that daily greps logins from syslog and which also checks timestamps on critical configuration files.

In addition there are plenty of tools for monitoring configuration file changes. Many Unix systems have an audit subsystem that can be enabled and which could also help.

Personally, I suspect that the best way to provide feedback in the event of a mistake is not to hunt for someone to pin blame on but to explain the problem to the whole group, explain why the change was a mistake and solicit ideas for preventing mistakes.

you should be able to determine who is becoming root by parsing /var/log/auth.log.

for example, if i su to root on my own box, a line like this is entered in /var/log/auth.log:

Oct 29 20:10:33 localhost su: pam_unix(su:session): session opened for user root by (uid=1000)

now looking in /etc/passwd, we see:

brad:x:1000:100:brad clawsie,,,:/home/brad:/bin/zsh

which lets us correlate uid 1000 to a name. writing a perl script to do this should be very straightforward, you're simply joining the uid across /var/log/auth.log and /etc/passwd over a certain time range, namely the mtime for the file you are investigating.

of course if someone su's to root and simply executes the touch command on the file you are researching, they will set its mtime, and you will falsely believe it is they who have edited the file.

if i were to recommend a best practice to avoid your issues in the first place, it would be to store all scripts, crons, config files...everything as files in a version control system that must be edited by a real user, then applied to the live system with some sort of deployment tool (make, apt, whatever). if people keep breaking things, have their changed signed off by a more senior developer before deployment.

your backups might not help you if you failed to make a copy at the right time...a version control system lets you revert back over as many edits as you like.

any time files on live production systems are being edited in place by root, thats a major problem. i can't tell in your question if hudson is a real person on your system or a synthetic user for a service (i.e. the hudson service). if it is a synthetic user, i would ask why this uid gets a "real" login shell at all.

Linux contains an advanced audit system, which one can set to track all changes to specific files.

From Understanding Linux Audit I see that it can be set up to also track SSH logins in addition to file changes, so it could be a possible solution to your problem.

Rename su to log_su then put actual su (now its log_su) call inside a wrapper named su

#!/usr/bin/env bash
$SU_LOG=<path to log file>
echo "User: $USER running su at $(date)" >> $SU_LOG
log_su

Both the su and sudo commands can be parametered to keep logs.

Assuming that (1) going to root is not a frequent action and that (2) usually not more than one admin becomes root at the same time, and given any file-modification time-stamp, your script can simply search in the su/sudo logs to locate the admin who was root at that time.

If you are using ssh, then you can generate an authentication key for each administrator and make them use it while logging in, rather than by the password mechanism.

As described in How can I get the comment of the current authorized_keys ssh key, one can use the comment field in the key-file to identify the logged-in person.

The rest is the same: Using the file modification time-stamp against auth.log to identify the presumed ssh session that edited that file.