5.5. Logging
During a user session, suSSHi creates three different kinds of log files.
5.5.1. System Log
The system log is a central log in which all system messages of all suSSHid daemons are recorded.
Messages vary from received connections and user login attempts to basic system messages like daemon startup.
As specified in the configuration SystemLogFile, system messages are written to the local file system.
Path variables
The path specified under SystemLogFile may contain the following tokens that are expanded at runtime once the user
has been authenticated at the gateway:
Token |
Description |
|---|---|
%d |
Day of month (01 … 31 |
%m |
Month (01 … 12) |
%y |
Year (2019 …) |
Note
The default path within the containers is /var/log/susshi/system/%y/%m/%d/system.log.
All system log messages are forwarded to Syslog as well.
The Syslog facility used for this messages can be set by the configuration file option SystemLogFacility.
The default facility is AUTH.
Please refer to the Partitions section for further details.
5.5.2. Session Logs
The session logs are individual logs per suSSHi user session and contain some details about what SSH protocol features
the user is making use of, what channels have been opened and closed and whether the requested features have been either
accepted or denied by suSSHi, etc.
The system messages are written to the local file system as specified in the configuration SessionLogFile.
Path variables
The path may contain the following tokens that are expanded at runtime once the user has been authenticated at the gateway:
Token |
Description |
|---|---|
%d |
Day of month (01 … 31 |
%m |
Month (01 … 12) |
%y |
Year (2019 …) |
%t |
Target ip address |
%u |
Gateway user ID |
%s |
suSSHid session identifier |
Note
The default path within the containers is /var/log/susshi/sessions/%y/%m/%d/%t/%u-%s.log.
Each entry in the session log is prefixed with the time of the event, the gateway user, client IP address, target user, target hostname and target IP address in the following form:
<time> susshid[<pid>]: <gateway_user>@<client_ip> ==> <target_user>@<target_host> (<target_ip>):
With this prefix it is very easy to grep through a bunch of log files and find the session you are looking for.
All session log messages are forwarded to Syslog as well.
The Syslog facility used for this messages can be set by the configuration file option SessionLogFacility.
The default facility is AUTH.
Please refer to the Partitions section for further details.
5.5.3. Audit Logs
The audit logs are the most advanced logs in suSSHi. They provide a detailed insight of what a user has done during a session, including full logging of interactive and command exec sessions in both directions:
the target server’s output, which contains all the data a server sends to the client, including the characters the user has typed unless echo has not been turned off on server side
the client user input, which contains any single keystroke typed by the user.
Furthermore, file transfers using SCP or SFTP can be logged so that it is clear which files have been touched and in which
way during the file transfer session.
Other logging like libpcap dumps of the forwarded IP traffic over a port forwarding channel or X11 session logs are
available as well.
Not all audit log files are ASCII readable but need some additional software to be read.
To understand audit logging, it is important to know that in a single SSH session, multiple channels can be requested by either the client or the target, and each channel can be of a different type. For example, the user can start an interactive login to a server and receive a shell I/O channel, but next request X11 or port forwarding or SFTP file transfer within the same SSH session, but on different channels.
To obtain a complete and detailed log of an SSH session, suSSHi examines each of these channels and logs the requests and traffic depending on what function the channel is used for. An interactive session results in some sort of ASCII logging of server output and user input, an SFTP protocol leads to a detailed logging of what the user has done within SFTP, etc.
Path variables
Audit messages are written to the local file system as specified in the configuration AuditLogFile.
The path may contain the following tokens that are expanded at runtime once the user has been authenticated at the gateway:
Token |
Description |
|---|---|
%d |
Day of month (01 … 31 |
%m |
Month (01 … 12) |
%y |
Year (2019 …) |
%t |
Target ip address |
%u |
Gateway user ID |
%s |
suSSHid session identifier |
%f |
File type |
Note
The default path within the containers is /var/log/susshi/audit/%y/%m/%d/%t/%u-%s.%f.
The file type (%f token) is expanded as shown in the following table.
The %f token must be specified otherwise reading the configuration file will fail.
Type |
Data Source |
Format |
Description |
|---|---|---|---|
agent |
Client & Target |
ASCII, line by line |
Auth Agent logging |
client |
Client |
ASCII |
Client input logging |
log |
Any |
ASCII, line by line |
Standard logfile |
pcap |
Client & Target |
Binary (PCAP) |
Packet capture file used by many utilities like tcpdump, wireshark etc. |
scp |
Client & Target |
ASCII, line by line |
SCP Logging |
session |
Target |
ASCII |
Server output logging |
sftp |
Client & Target |
ASCII, line by line |
SFTP Logging |
time |
Client & Target |
ASCII |
Timing file for server output and client input logging |
x11 |
Client & Target |
Binary (PCAP) |
X11 data logging in PCAP format |
Please refer to the Partitions section for further details.
5.5.3.1. Audit Logging in Detail
With suSSHi it is possible to log all kinds of different SSH protocol features in the file system according to the protocol. The different protocol characteristics require suitable log file formats to write the output files for later analysis. The audit log files are (usually) not typical log files with line-by-line events, but special files.
5.5.3.1.1. Interactive Session Logging
Server Output
If a user logs on to a target server and no optional command is specified by the user, a default interactive shell runs
on the remote server.
If the client specified a command but requested a PTY (for example, with the -t option in the OpenSSH ssh client),
audit logging also becomes this type of session logging.
The interactive logging of the session server output generates two files:
The first file contains all output sent by the server without filtering or losing a single character.
The second file contains time information about the server output.
This allows the output of the server session to be reproduced 100% in the correct order and at the correct time since the data originally arrived.
Client Input
In the server output log (see above), only the output generated by the server is logged. So if the terminal is set to “no output” (e.g. when entering a password or forced by the user by sending the corresponding terminal control sequences), no output is included in the server output logs. To obtain all data entered by the user, client input logging can be enabled by setting the appropriate logging mask.
In the interactive session client input log, suSSHi also generates two different files:
One with the input entered by the user
One with the timing information as described above.
If server output logging is also enabled, the time information together with the server output timing go to the same time file.
5.5.3.1.2. Remote Command Execution
Whenever a user instructs his SSH client to execute a remote command at login (e.g. ssh gwuser@root@server@susshi shutdown -h now), suSSHi inspects the session as an exec session instead of an interactive session. Since execution sessions are very often used for remote file copying, and this could naturally fill a journaling file very quickly (think of an user copying an entire DVD by using a tar or cpio pipe over an SSH session), it is important to have a different journaling strategy for the data transferred over these sessions. Only when browsing the command to be executed on the target, it is simply impossible for suSSHi to guess whether the resulting data should be logged or simply ignored to save disk space.
The remote command logging algorithm looks like this:
When suSSHi receives a PTY request, the logging is converted to standard interactive session logging, since we now have some kind of interactive I/O that requires a PTY. The user can also request a PTY when executing remote commands (e.g. with the
-toption in OpenSSH’s ssh client). This is sometimes done to run a preferred user shell instead of the default user shell in the user profile (e.g. ssh -t gwuser@root@server@susshi bash).If the remote command is a form of SCP call, the logging is converted to SCP logging (see below).
If the remote command is retrieved from the user-definable stop patterns (see the
Execution Stop Patternin Partitions configuration), no audit log of the session is performed. It can be very helpful for the administrator to observe the session logs from time to time to look for patterns that need to be added to the configuration to save disk space by logging large file transfers.In all other cases, suSSHi always logs the number of bytes configured with the exec log max-size (see Partitions) and then stops logging to save disk space. If the value is not set,
200MBis the default.
5.5.3.1.3. SFTP
All SCP traffic is analyzed in its function and logged in a human-readable log file. The SCP protocol is very restrictive in the supported functions: a file can be copied or a directory (which is always a recursive copy). There are no functions like file deletion, etc. that you can find in SFTP. To enable SCP audit logging, the logging mask must be set accordingly.
5.5.3.1.4. Port Forwarding
Any port forwarding, whether local or remote, can only transfer TCP traffic. If the logging mask is set accordingly, all traffic on the forwarded port is logged into a libpcap-compatible packet capture which can be read with any software such as tcpdump or Wireshark that can read libpcap-formatted files.
Since the port forwarding within the SSH protocol is ultimately only a bidirectional data stream (socket IO), the actual IP addresses of the endpoints communicating over these sockets cannot be determined. Therefore, always check the session log and be careful when using IP addresses and ports in forensics without performing a thorough traffic analysis.
For output in PCAP format, the IP addresses are therefore always set to 1.1.1.1 for the client side and 2.2.2.2
for the target side.
By the way, this also applies if IPv6 is actually used to access the socket.
5.5.3.1.5. X11
X11 logging is essentially the same as standard port forwarding logging.
Traffic is logged in a libpcap-compatible file with the X11 file type set.
The same applies for IP addresses as for standard port forwarding.
The port is always set to 6000, regardless of which display the X server is running on.
The X11 libpcap files can be read by any software that can read libpcap files for further analysis.