10.5. Restoring and Recovering Resources
During active incident response, there should be considerations
toward working on recovery. The actual breach will dictate the course
of recovery. This is when having backups or offline, redundant systems
will prove invaluable. For recovery, the response team should be
planning to bring back online any downed systems or applications, such
as authentication servers, database servers, and any other production
resources.
Having production backup hardware ready for use is highly
recommended, such as extra hard drives, hot-spare servers, and the like.
Ready-made systems should have all production software loaded and ready
for immediate use. Perhaps only the most recent and pertinent data
would need to be imported. This ready-made system should be kept
isolated from the rest of the potentially affected network. If a
compromise occurs and the backup system is a part of the network, then
the purpose of having a backup system is defeated.
System recovery can be a tedious process. In many instances there
are two courses of action from which to choose. Administrators can
perform a clean reinstallation of the operating system followed by
restoration of all applications and data. Alternatively, administrators
can patch the system of the offending vulnerability and bring the
affected system(s) back into production.
10.5.1. Reinstalling the System
Performing a clean reinstallation ensures that the affected system
will be cleansed of any trojans, backdoors, or malicious
processes. Reinstallation also ensures that any data (if restored from
a trusted backup source) is cleared of any malicious modification.
The drawback to total system recovery is the time involved in
rebuilding systems from scratch. However, if there is a
hot backup system available for use where the
only action to take is to dump the most recent data, then system
downtime is greatly reduced.
10.5.2. Patching the System
The alternate course to recovery is to patch the affected
system(s). This method of recovery is more dangerous to perform and
should be undertaken with great caution. The danger with patching a
system instead of reinstalling is determining whether or not you have
sufficiently cleansed the system of trojans,
holes, and corrupted data. If using a modular kernel, then patching a
breached system can be even more difficult. Most
rootkits (programs or packages that a cracker
leaves to gain root access to your system), trojan system commands,
and shell environments are designed to hide malicious activities from
cursory audits. If the patch approach is taken, only trusted binaries
should be used (for example, from a mounted, read-only CD-ROM).
