Hello all,
Just wanted to let people know I have released a USN Journal record parser plugin for volatility.
https://github.com/tomspencer/volatility/tree/master/usnparser
For anyone who wants a refresher on the USN journal and its forensic significance, I recommend these <http://journeyintoir.blogspot.com/2013/01/re-introducing-usnjrnl.html> sites <http://securitybraindump.blogspot.com/2011/07/dear-diary-today-i-was-infect…> . The USN journal has been very useful to us in a variety of circumstances, so I highly recommend including it in your work/timelining flow if you dont already. Ive even seen situations where journal records from external NTFS volumes persist in memory even after the device is removed, but those results have been inconsistent.
This plugin should work on any version of Windows that does USN journaling (Vista/2008 and up, XP SP3 if you specifically turn it on), and it should play well with Unicode file names including calling out special Unicode characters like directional changes.
For an example of calling out direction changes:
2014-04-16 20:51:23.116,0x257L,0x3L,0x14de3L,0x3L,0x4806360L,"utf-canary.<RLO>txt.scr",RENAME_NEW_NAME & CLOSE,ARCHIVE
The <RLO> indicates a Unicode Right-To-Left Override character. This means that although the actual filename extension was txt.scr, a screen saver filetype which is expected to be an executable, it would have appeared to the user as utf-canary.rcs.txt which most users would expect to be a text file they could safely click.
This is my first volatility plugin so Id love any feedback/suggestions/questions. Based on some feedback from MHL I'll be doing some testing to see if I should turn on timestamp validation by default, but I'm open to other changes/additions people would like to see.
Basic notes on the plugin are below. Enjoy!
Thanks,
Tom
A note on USN record versions. The literature I can find seems to suggest that starting with Windows NT 6.2 families (8/2012) the OS should be using version 3 records, and this plugin does support these records. That said, testing has shown that at least in memory, these OSs still seem to use v2 records. As such, unless otherwise specified with the -R3 flag, this plugin will always assume v2 records.
In my testing there has been a significant number of duplicates in memory, so piping to sort u can be effective.
Invocation example:
CSV output
$ vol.py --profile Win7SP1x64 -f Windows7SP1x64.vmem usnparser --output=csv
Volatility Foundation Volatility Framework 2.3.1
timestamp,MFTEntry,MFTEntryUSN,Parent,ParentUSN,usn#,"Filename",Reason,Attributes
2014-01-26 09:01:19.079,0x5056L,0x1L,0x7beL,0x1L,0x10f35c0L,"ngen_service.log",EXTEND & CLOSE,ARCHIVE
2014-01-26 09:01:19.079,0x7d8L,0x86L,0x7beL,0x1L,0x10f3620L,"ngen_service.lock",CREATE & DELETE & CLOSE,ARCHIVE
2014-01-26 09:01:19.142,0x7d8L,0x89L,0x28eL,0x1L,0x10f39d8L,"GACLock.dat",CREATE,ARCHIVE & TEMPORARY
2014-01-26 09:01:19.142,0x7d8L,0x89L,0x28eL,0x1L,0x10f3a30L,"GACLock.dat",CREATE & DELETE & CLOSE,ARCHIVE & TEMPORARY
....
BODY output, suitable for using with mactime
$ vol.py --profile Win7SP1x64 -f Windows7SP1x64.vmem usnparser --output=body | head
Volatility Foundation Volatility Framework 2.3.1
0|[USN JOURNAL] ngen_service.log EXTEND & CLOSE/USN: 17774016/PARENT MFT: 1982|20566|---a-----------|0|0|0|1390726879|1390726879|1390726879|1390726879
0|[USN JOURNAL] ngen_service.lock CREATE & DELETE & CLOSE/USN: 17774112/PARENT MFT: 1982|2008|---a-----------|0|0|0|1390726879|1390726879|1390726879|1390726879
0|[USN JOURNAL] GACLock.dat CREATE/USN: 17775064/PARENT MFT: 654|2008|---a--t--------|0|0|0|1390726879|1390726879|1390726879|1390726879
0|[USN JOURNAL] GACLock.dat CREATE & DELETE & CLOSE/USN: 17775152/PARENT MFT: 654|2008|---a--t--------|0|0|0|1390726879|1390726879|1390726879|1390726879
The current version's help output (flags specific to usnparser start at "-T, --timestamp"):
$ vol.py usnparser -h
Volatility Foundation Volatility Framework 2.3.1
Usage: Volatility - A memory forensics analysis platform.
Options:
-h, --help list all available options and their default values.
Default values may be set in the configuration file
(/etc/volatilityrc)
--conf-file=/home/vol/.volatilityrc
User based configuration file
-d, --debug Debug volatility
--plugins=PLUGINS Additional plugin directories to use (colon separated)
--info Print information about all registered objects
--cache-directory=/home/vol/.cache/volatility
Directory where cache files are stored
--cache Use caching
--tz=TZ Sets the timezone for displaying timestamps
-f FILENAME, --filename=FILENAME
Filename to use when opening an image
--profile=WinXPSP2x86
Name of the profile to load
-l LOCATION, --location=LOCATION
A URN location from which to load an address space
-w, --write Enable write support
--dtb=DTB DTB Address
--output=text Output in this format (format support is module
specific)
--output-file=OUTPUT_FILE
write output in this file
-v, --verbose Verbose information
--shift=SHIFT Mac KASLR shift address
-g KDBG, --kdbg=KDBG Specify a specific KDBG virtual address
-k KPCR, --kpcr=KPCR Specify a specific KPCR address
-T, --timestamp Print timestamps instead of human-readable dates
-E, --unixtime Use Unix Epoch 32-bit timestamps instead of native
Windows 64-bit timestamps (loses subsecond accuracy).
DOES NOT imply -T above.
-C, --checktime Don't show entries with timestamps outside of unix
epoch range to reduce corrupt entries
-S, --strict Enable stricter checks on record integrity to further
reduce corrupt entries
-O, --offset Show the physical offset for each record
-R RECORDTYPE, --recordtype=RECORDTYPE
Force version of USN record (2 or 3) to search for. In
testing so far all OS's seem to use version 2 records
in memory (even 8.1/2012r2 which purport to use R3).
As such, default is R2.
-U, --unicode Show unicode (utf-8) filenames. Be aware that due to
corrupted records there will likely be strange
characters in some places. Using -C and -S can help
cut this down.
---------------------------------
Module USNParser
---------------------------------
Scans for and parses USN journal records
Hi,
According to Volatility issue #383 'tmpfs' extraction doesn't work because Volatility doesn't support NUMA systems.
Question 1 - Is it on the roadmap for future versions?
I deal primarily with Multi-CPU cloud systems so this is definitely a desired feature.
Question 2- Is it reasonably feasible to manually extract tmpfs from a system RAM dump?
Following the 'linux_tmpfs' module through the debugger showed that it was able to locate the /dev/shm tmpfs file system (replicating 2 levels in my output directory), it just croaked when it came time to retrieve the actual file data.
I figure that if I can manually determine whatever offset it needs then I can set the proper variable in a debug session.
Any thoughts?
Thanks,
Geoff
==============================
Geoff Torres HP Global Cyber Security
8000 Foothills Blvd.
Roseville, CA. 95747
916-785-3323
==============================
Hello
I've been testing volatility and looking through the results. In
particular, within the Handles extraction, I found the following line...
0xfffffa8009648800 3544 0x1a78 0x120089 File
\Device\TrueCryptVolumeK\Test.txt
This is a file that I had stored in a hidden volume. I attempted to
re-create this type of entry with 3 further memory dumps with no such
success (No files within TrueCrypt volume). Can anyone advise why this
filename "Test.txt" was found? I see that a lot of files can be found in
the Handles extraction, but haven't been able to find any documentation on
how files are included in this section.
I ran the following command on an 8GB Memory dump which was captured via
FTK Imager...
vol.exe -f memdump.mem --profile=Win7SP1x64 --output=text
--output-file=handles-files.txt handles -t File
This result was a total surprise to find. In further testing, I attempted
to do the following within the hidden volume...
- Create new files
- Copy files into the volume
- Leave files open while closing the volume within TrueCrypt
Thanks,
R
Hello again,
So, now that I am using the right profile, the plug ins seem to work.
My goal is recovering unsaved notepad files from hibernation. I have a hiberfil.sys from a Win 7 SP1 64 bit system.
My next step seemed to be using pslist to get the PIDs, and putting those into one of the built in plugins.
I've tried dumpfiles, vaddump, memdump, and some others.
It looks like I should be able to piece something together between the results of dumpfiles with a PID switch, and of vaddump with a PID switch. I haven't figured that out yet. I'm wondering if there is a more specific switch. They both seem to produce a lot more files than I need.
Is there a better way to use volatility's built in tools to pull out files from notepad?
Is there an add on that I can download which will pull out something more quickly and cleanly?
Thanks,
andybellman(a)outlook.com
Good afternoon,
I'm having a bit of trouble using Volatility for memory forensics with the
goal of malware detection. I've captured a memory dump of a Windows 7 SP1
x64 machine using winpmem_1.5.5.exe and am using the 2.3.1 standalone
variant of Volatility on a Windows 7 SP1 x64 machine. When i issue commands
such as 'connections' , 'connscan' , 'sockets' i get the error "This
command does not support the profile Win7SP1x64." I've also tried
Volatility Standalone 2.3 and 2.2. Any explanation would be greatly
appreciated. Thanks!
Hi,
is anybody else also running into issues when using mftparser on Win7 64
Bit?
I get the following:
WARNING : volatility.obj : Cant find object NullString in profile
<volatility.plugins.overlays.windows.win7.Win7SP1x64 object at 0xb4fc44c>?
Which results in entries like:
(FN)
0x184000|None\None\None\None\None\None\None|153380|---a-----------|0|0|480|1336379877|1336379877|1336379877|1336379877
Thanks,
Markus
I am having problems with my PSXView. On multiple occasions I have
started this command and left it running overnight and by the next
morning there has been no reported data. The command appears to be
stalled. I am not sure where to look for the exact problem. I have
looked into the Python address space with WinDbg and have noted, with
!VAD, a segment of memory that is EXECUTE_READWRITE that is not listed
as a process. It is identified as "Private". When peering into the
segment of memory, I have noted a number of locations where there is
an "MZ" prefix that designates a Windows PE. This is followed by
"This program cannot......" so I know that this block contains
executable code. When analyzing the code further there is a number of
these programs, I have found headers designating that these are DLLs.
Should this code block be present? With my limited training, I
understand that all DLLs should be loaded by the loader and reflected
in the address space with a VAD not burried in a segment of code. Has
anyone else experienced this problem? Is my PSXView problem related
to something else? Is there a way to isolate the issue further from
here? I did a dump of Python using Procexedump but have not reviewed
the IAT of the file or attempted to disassemble the file. I am new to
reverse engineering so I am looking for the closest rope to grab onto.
Hi all,
I've followed the documentation to first dump the memory device cross
compiling lime and then creating the profile for a linux device on arm.
Unfortunately I wasn't able to use volatility on the memory dump.
I'm using volatility 2.3.1, the kernel is a linux vanilla 2.6.31.14 + a
custom grsecurity+pax configuration.
Below some output from the commands, any suggestion on next step to
troubleshoot where is the problem ?
boos@vnoise:~/Downloads/volatility-2.3.1$ python vol.py --info | grep
Profile | grep Linux
Volatility Foundation Volatility Framework 2.3.1
LinuxTESTARM - A Profile for Linux TEST ARM
$ python vol.py -f /home/boos/arm-mem-image imageinfo
Determining profile based on KDBG search...
Suggested Profile(s) : No suggestion (Instantiated with
LinuxUbuntu1204x64)
AS Layer1 : LimeAddressSpace (Unnamed AS)
AS Layer2 : FileAddressSpace (/home/boos/arm-mem-image)
PAE type : No PAE
DTB : 0x1c0d000L
Traceback (most recent call last):
File "vol.py", line 184, in <module>
main()
File "vol.py", line 175, in main
command.execute()
File "/home/boos/Downloads/volatility-2.3.1/volatility/commands.py", line
122, in execute
func(outfd, data)
File
"/home/boos/Downloads/volatility-2.3.1/volatility/plugins/imageinfo.py",
line 36, in render_text
for k, v in data:
File
"/home/boos/Downloads/volatility-2.3.1/volatility/plugins/imageinfo.py",
line 93, in calculate
kdbgoffset = volmagic.KDBG.v()
File "/home/boos/Downloads/volatility-2.3.1/volatility/obj.py", line 737,
in __getattr__
return self.m(attr)
File "/home/boos/Downloads/volatility-2.3.1/volatility/obj.py", line 719,
in m
raise AttributeError("Struct {0} has no member
{1}".format(self.obj_name, attr))
AttributeError: Struct VOLATILITY_MAGIC has no member KDBG
boos@vnoise:~/Downloads/volatility-2.3.1$ python vol.py --profile
LinuxTESTARM -f /home/boos/arm-mem-image linux_dmesg
Volatility Foundation Volatility Framework 2.3.1
No suitable address space mapping found
Tried to open image as:
MachOAddressSpace: mac: need base
LimeAddressSpace: lime: need base
WindowsHiberFileSpace32: No base Address Space
WindowsCrashDumpSpace64: No base Address Space
HPAKAddressSpace: No base Address Space
VirtualBoxCoreDumpElf64: No base Address Space
VMWareSnapshotFile: No base Address Space
WindowsCrashDumpSpace32: No base Address Space
AMD64PagedMemory: No base Address Space
IA32PagedMemoryPae: No base Address Space
IA32PagedMemory: No base Address Space
MachOAddressSpace: MachO Header signature invalid
MachOAddressSpace: MachO Header signature invalid
LimeAddressSpace: Invalid Lime header signature
WindowsHiberFileSpace32: PO_MEMORY_IMAGE is not available in profile
WindowsCrashDumpSpace64: Header signature invalid
HPAKAddressSpace: Invalid magic found
VirtualBoxCoreDumpElf64: ELF64 Header signature invalid
VMWareSnapshotFile: Invalid VMware signature: 0x0
WindowsCrashDumpSpace32: Header signature invalid
AMD64PagedMemory: Incompatible profile LinuxTESTARM selected
IA32PagedMemoryPae: Failed valid Address Space check
IA32PagedMemory: Failed valid Address Space check
FileAddressSpace: Must be first Address Space
ArmAddressSpace: Failed valid Address Space check
--
Roberto Martelloni
boos @ http://boos.core-dumped.info
