Posts Tagged ‘Convert to CSV’

Articles

Backing up/Archiving Server Security Event logs to CSV

In Uncategorized on October 16, 2010 by brwilkinson Tagged: , , , , , , , ,

Okay first of all this script can backup any Event logs, even custom logs, however I wrote it for the Security Event Logs.

I recommend running this locally, especially if you are on a large domain with a lot of users/logins, which mean the Security Event Logs fill up pretty fast.

I designed this to be used via PS-Remoting, where it runs via a scheduled task from a central server and remotely runs on all Domain Controllers on the network.

It archives the logs into folders based on the date and you can easily adjust the file formatting to suit your needs.

I have recently changed the code to only backup the following items:

As a suggestion for all values, maybe: EventID, MachineName, Category, CategoryNumber, EntryType, ReplacementStrings, TimeGenerated, UserName

The script just stores the replacementStrings rather than the Message. This is mainly due to the fact that the message is hard to Parse afterwards, since it contains whitespace and takes up many lines.

To explore what is possible I suggest you run the following and select the items you need when performing the Export-Csv.

$a = get-eventlog -log Security -newest 1            
$a | format-list -property *            
$a | Export-Csv $PathLocal
<#
EventID            : 538
MachineName        : TESTDC009
Data               : {}
Index              : 967753
Category           : Logon/Logoff
CategoryNumber     : 2
EntryType          : SuccessAudit
Message            : User Logoff:
                     
                         User Name:    userb01
                     
                         Domain:        YourDomain
                     
                         Logon ID:        (0x0,0x1132C3D5)
                     
                         Logon Type:    3
                     
Source             : Security
ReplacementStrings : {userb01, YourDomain, (0x0,0x1132C3D5), 3}
InstanceId         : 538
TimeGenerated      : 8/3/2010 12:19:34 AM
TimeWritten        : 8/3/2010 12:19:34 AM
UserName           : YourDomain\userb01
Site               : 
Container          :
#>

So the function of the script is pretty simple and the full script is available for download on TechNet Script Center Repository.

#Requires -Version 2.0                        
[CmdletBinding()]                        
 Param                         
   (                        
    [Parameter(Mandatory=$false,                        
               ValueFromPipeline=$true,                        
               ValueFromPipelineByPropertyName=$true)]                        
    [String]                        
    $computer = "$ENV:COMPUTERNAME",                                    
    [ValidateSet("Application", "Security", "System")]                        
    [Alias("l","log")]                        
    [String]                        
    $LogName = "Security",                        
    [String]$BackupLocal = "D:\BackupLOGS",                        
    [String]$Backupremote = "\\server\logs\seclogs"                                   
   )#End Param                        
Begin                        
{                        
 Write-Host "Processing $Logname Logs .. .. Server:"$computer (get-date)                        
}                        
Process                        
{                        
    $BaseDirLocal = "$BackupLocal\{0:yyyy_MM}-Logs" -f [DateTime]::now                        
    $LogFileName = "{0}-{1:yyyyMMdd_HHmm}-{2}.csv" `
                                -f $Computer,[DateTime]::now,$LogName                        
    $PathLocal = Join-Path -Path $BaseDirLocal -ChildPath $LogFileName                        
    Write-Host "  + Processing $LogName Log"                        
    Write-Host "    - Reading $LogName Log"                        
                            
    #Save the logs from the Event Logs to PSObject/Memory                        
    $SecLogs = get-eventlog -LogName $LogName                        
                            
    #Clear the event logs                        
    Write-Host "    - Clearing $LogName Log"                        
    Clear-EventLog -LogName $LogName                        
                            
    # Make sure the local directory exists                        
    If(!(Test-Path $BaseDirLocal))                        
    {                        
      New-Item $BaseDirLocal -type Directory -force | out-Null                        
    }                        
    Write-Host "    - Writing to CSV"                        
                            
    # Export from PSObject/Memory to CSV                         
    $SecLogs | ForEach-Object {$RString = $_.ReplacementStrings |             
       ForEach-Object {$_}                        
    $Hash = @{                                     
    EventID       =$_.EventID                        
    MachineName   =$_.MachineName                        
    Category      =$_.Category                        
    CategoryNumber=$_.CategoryNumber                        
    EntryType     =$_.EntryType                        
    ReplStrings   ="$RString"                        
    TimeGenerated =$_.TimeGenerated                        
    UserName      =$_.UserName}                        
    New-Object PSObject -Property $Hash                        
    } | Export-Csv -Path $PathLocal -NoTypeInformation                        
    Write-Host "    - Writing to CSV complete"                        
                            
    # Copy the Logs/CSV up to the fileshare                        
    $BaseDirRemote = "$Backupremote\{0:yyyy_MM}-Logs" -f [DateTime]::now                        
    If(!(Test-Path -Path $BaseDirRemote))                        
    {                        
      New-Item $BaseDirRemote -type Directory -force | out-Null                        
    }                        
                            
    # Remove the Old File                        
    If(Test-Path -Path $BaseDirLocal)                        
    {                        
      # Move all files from the local directory, then delete the directory                        
      Write-Host "    - Archiving CSV file: $logFile"                        
      Move-Item "$BaseDirLocal\*" -Destination $BaseDirRemote -Force                        
      Remove-Item -Path $BaseDirLocal                        
    }                        
}                        
End                        
{                        
 $Msg = "Processing $LogName log now complete, {0} logs exported."`
                                                       -f $SecLogs.count                        
 $Msg2 = "Logs were saved to {0}\{1}" -f $BaseDirRemote, $LogFileName                        
 Write-Host $Msg                        
 Write-Host $Msg2                        
 (get-date)                        
}

Summary Info

As I mentioned above I recommend running this script remotely via PS-Remoting. I will provide some information on configuring a list of domain computers for PS-Remoting in a later post. Actually what is being configured is Windows Remote Management (WinRM), via Web Service Management (WS-Management).

In summary you may wish to run the above script remotely and you would use a script that looks like the one below.

Executing a Script by PS Remoting:

# List the hosts you want to run the remote session                        
$Hosts = "adtestDC03","adtestDC04","adtestDC02"            
# Start the session                        
$sessions = New-PSSession –computerName $Hosts                         
# Run the commands via the Script            
$Path = "C:\PS\Export-Logs\Start-BackupEventLogs.ps1"            
$job = Invoke-Command -Session $sessions -filepath $Path –AsJob                         
# Monitor the job and then report the information back once its complete                        
while ((Get-Job).State -eq "Running")                        
{                        
 write-host ""                        
 Write-Host ("*"*40)                        
 write-host "Job running -" (Get-Date)                        
 # The jobs can be viewed to ensure they completed correctly                        
 Get-Job | ft Id,Name,State,Location  -AutoSize                        
 # If the task takes a while you can sleep for a while each cycle                        
 Sleep 45                        
}                          
# If there are any results from your script, you can view the output                        
$results = Receive-Job $job                        
$results                        
# Cleanup the Sessions                        
Remove-PSSession -Session $sessions                        
# Cleanup the jobs                        
Get-Job | Remove-Job

Updated Script

Not long after running this script I found that the size of the CSV files that were being backed up were growing very fast, so I updated the script to ZIP the Event logs up prior to copying them up to the server.

I haven’t updated the whole script here, however I used the PSCX zip function to perform the zip and I have added two code sections below for an example. The compression was excellent.

Begin            
{            
 Import-Module PSCX            
 Write-Host "Processing $Logname Logs .. .. Server:"$computer (get-date)            
}            
Process            
{            
        # Some more code here            
                    
        # Export from PSObject/Memory to CSV             
        $SecLogs | ForEach-Object {$RString = $_.ReplacementStrings |             
                                                ForEach-Object {$_}            
        $Hash = @{                         
        EventID       =$_.EventID            
        MachineName   =$_.MachineName            
        Category      =$_.Category            
        CategoryNumber=$_.CategoryNumber            
        EntryType     =$_.EntryType            
        ReplStrings   ="$RString"            
        TimeGenerated =$_.TimeGenerated            
        UserName      =$_.UserName}            
        New-Object PSObject -Property $Hash            
        } | Export-Csv -Path $PathLocal -NoTypeInformation            
        Write-Host "    - Writing to CSV complete"            
                    
        # Zip up the Logs/CSV File            
        Write-Host "    - Converting to Zip"            
        $ZipLogArchiveFile = Get-ChildItem $PathLocal | write-zip -level 9            
        if (Test-Path -Path $ZipLogArchiveFile)            
        {            
         Remove-Item -Path $PathLocal            
        }            
                    
        # Copy the Logs/CSV up to the fileshare            
        $BaseDirRemote = "$Backupremote\{0:yyyy_MM}-Logs" -f [DateTime]::now            
        If(!(Test-Path -Path $BaseDirRemote))            
        {            
          New-Item $BaseDirRemote -type Directory -force | out-Null            
        }            
                    
        # Remove the Old File            
        If(Test-Path -Path $BaseDirLocal)            
        {            
          # Move all files from the local directory, then delete            
          Write-Host "    - Archiving Zip file:"$ZipLogArchiveFile.Name            
          Move-Item "$BaseDirLocal\*.zip" -Destination $BaseDirRemote -Force            
          Remove-Item -Path $BaseDirLocal            
        }            
                    
        # Some more code here            
 }            
 End            
 {            
    # Some more code here            
 }