Hey Nir,
We can definitely help out with integrating your code with the object
system. I was just in the process of testing it for the first time this
afternoon.
Here are a few details (I'll copy them to the issue tracker in a sec).
Basically I started with an ESX 4.1.0 and grabbed the following:
* vmsn from xpsp2 x86 256 MB
* vmsn from win7 sp1 x86 512 MB
* vmss from server 2008 sp1 x64 4 GB
Your AS with the latest 2.1 alpha (so about r1983) worked fine for the xp
vmsn. (by fine I mean pslist worked, but other plugins may not work
properly due to what scudette said about zread)
=================================================
VMSN - XPSP3 x86 @ 256 MB RAM
$ python vol.py -d -f Andrew-Snapshot6.vmsn pslist
Volatile Systems Volatility Framework 2.1_alpha
[snip]
DEBUG : volatility.utils : Trying <class
'volatility.plugins.addrspaces.vmware.VMWareSnapshotFile'
DEBUG : volatility.plugins.addrspaces.vmware:
RegionCount: 1
DEBUG : volatility.plugins.addrspaces.vmware: Virtual Address:
0, Physical Address: 0, Size: 10000000
DEBUG : volatility.plugins.addrspaces.vmware: dtb: 39000
DEBUG : volatility.utils : Succeeded instantiating
<volatility.plugins.addrspaces.vmware.VMWareSnapshotFile object at
0x10363a890
Offset(V) Name
PID PPID Thds Hnds Sess Wow64
Start Exit
---------- -------------------- ------ ------ ------ -------- ------ ------
-------------------- --------------------
0x81bcc830 System 4 0 52 477 ------ 0
0x8194a020 smss.exe 364 4 3 21 ------ 0
2012-01-25 20:44:20
0x81954020 csrss.exe 616 364 10 345 0 0
2012-01-25 20:44:20
0x81951128 winlogon.exe 640 364 16 495 0 0
2012-01-25 20:44:20
0x81a897a8 services.exe 684 640 15 272 0 0
2012-01-25 20:44:20
[snip]
It also worked fine for the win7 vmsn:
=================================================
VMSN - Windows 7 SP0 x86 @ 512 MB RAM
$ python vol.py -d -f Abraham-Snapshot2.vmsn --profile=Win7SP0x86 pslist
Volatile Systems Volatility Framework 2.1_alpha
[snip]
DEBUG : volatility.utils : Trying <class
'volatility.plugins.addrspaces.vmware.VMWareSnapshotFile'
DEBUG : volatility.plugins.addrspaces.vmware:
RegionCount: 1
DEBUG : volatility.plugins.addrspaces.vmware: Virtual Address:
0, Physical Address: 0, Size: 20000000
DEBUG : volatility.plugins.addrspaces.vmware: dtb: 185000
DEBUG : volatility.utils : Succeeded instantiating
<volatility.plugins.addrspaces.vmware.VMWareSnapshotFile object at
0x102a1a750
Offset(V) Name
PID PPID Thds Hnds Sess Wow64
Start Exit
---------- -------------------- ------ ------ ------ -------- ------ ------
-------------------- --------------------
0x83f2f730 System 4 0 93 494 ------ 0
2012-03-15 15:04:12
0x84f32c48 smss.exe 252 4 2 29 ------ 0
2012-03-15 15:04:12
0x85708d40 csrss.exe 364 356 9 386 0 0
2012-03-15 15:04:48
0x82050030 wininit.exe 400 356 3 75 0 0
2012-03-15 15:04:48
0x856d9370 csrss.exe 408 392 7 201 1 0
2012-03-15 15:04:48
0x8207f030 services.exe 468 400 8 198 0 0
2012-03-15 15:04:49
0x8208e030 lsass.exe 476 400 8 711 0 0
2012-03-15 15:04:49
[snip]
I then reproduced the same thing Jesse is seeing on the server 2008 x64 w/
4 GB:
=================================================
VMSS - Server 2008 SP1 x64 @ 4 GB RAM
$ python vol.py -d -f Win2008SP1x64-9de64630.vmss --profile=Win2008SP1x64
pslist
Volatile Systems Volatility Framework 2.1_alpha
[snip]
DEBUG : volatility.utils : Trying <class
'volatility.plugins.addrspaces.hibernate.WindowsHiberFileSpace32'
DEBUG : volatility.utils : Trying
<class
'volatility.plugins.addrspaces.crash.WindowsCrashDumpSpace64'
DEBUG : volatility.utils : Trying
<class
'volatility.plugins.addrspaces.crash.WindowsCrashDumpSpace32'
DEBUG : volatility.utils : Trying
<class
'volatility.plugins.addrspaces.vmware.VMWareSnapshotFile'
DEBUG : volatility.plugins.addrspaces.vmware:
Read region count from
file: 2
DEBUG : volatility.plugins.addrspaces.vmware: RegionCount: 2
DEBUG : volatility.plugins.addrspaces.vmware: Virtual Address:
0, Physical Address: 0, Size: C0000000
Virtual Address: 100000000, Physical Address: C0000000, Size: 40000000
DEBUG : volatility.plugins.addrspaces.vmware: dtb: 124000
DEBUG : volatility.utils : Succeeded instantiating
<volatility.plugins.addrspaces.vmware.VMWareSnapshotFile object at
0x102a1a750
DEBUG :
volatility.utils : Voting round
DEBUG : volatility.utils : Trying <class
'volatility.plugins.addrspaces.hibernate.WindowsHiberFileSpace32'
DEBUG : volatility.utils : Trying
<class
'volatility.plugins.addrspaces.crash.WindowsCrashDumpSpace64'
DEBUG : volatility.utils : Trying
<class
'volatility.plugins.addrspaces.crash.WindowsCrashDumpSpace32'
DEBUG : volatility.utils : Trying
<class
'volatility.plugins.addrspaces.vmware.VMWareSnapshotFile'
DEBUG : volatility.utils : Trying
<class
'volatility.plugins.addrspaces.amd64.AMD64PagedMemory'
DEBUG : volatility.utils : Succeeded
instantiating
<volatility.plugins.addrspaces.amd64.AMD64PagedMemory object at 0x1036c91d0
DEBUG : volatility.utils : Voting round
DEBUG : volatility.utils : Trying <class
'volatility.plugins.addrspaces.hibernate.WindowsHiberFileSpace32'
DEBUG : volatility.utils : Trying
<class
'volatility.plugins.addrspaces.crash.WindowsCrashDumpSpace64'
DEBUG : volatility.utils : Trying
<class
'volatility.plugins.addrspaces.crash.WindowsCrashDumpSpace32'
DEBUG : volatility.utils : Trying
<class
'volatility.plugins.addrspaces.vmware.VMWareSnapshotFile'
DEBUG : volatility.utils : Failed
instantiating (exception): unpack
requires a string argument of length 4
DEBUG : volatility.utils : Trying <class
'volatility.plugins.addrspaces.amd64.AMD64PagedMemory'
DEBUG : volatility.utils : Trying
<class
'volatility.plugins.addrspaces.intel.JKIA32PagedMemory'
DEBUG : volatility.utils : Trying
<class
'volatility.plugins.addrspaces.legacyintel.IA32PagedMemoryPae'
DEBUG : volatility.utils : Trying
<class
'volatility.plugins.addrspaces.intel.JKIA32PagedMemoryPae'
DEBUG : volatility.utils : Trying
<class
'volatility.plugins.addrspaces.legacyintel.IA32PagedMemory'
DEBUG : volatility.utils : Trying
<class
'volatility.plugins.addrspaces.standard.FileAddressSpace'
Offset(V) Name PID
PPID Thds Hnds
Sess Wow64 Start Exit
------------------ -------------------- ------ ------ ------ --------
------ ------ -------------------- --------------------
0xfffffa8003ca8950 System 4 0 104 496
------ 0 2012-03-02 07:16:23
/Users/Michael/volatility_pe_exceptions/volatility/plugins/overlays/windows/windows.py(262)windows_to_unix_time()
-> unix_time = windows_time / 10000000
(Pdb) windows_time
<NoneObject: Unable to read 8 bytes from 18446738026473744904
You can see the error occurred because windows_time at 0xfffffa8004a86a08L
(hex value of the decimal offset above) could not be fetched from the vmss
file. Since it appears the System process is able to be found, we should be
able to break into a volshell (which uses the System process AS by default)
and try some checks:
$ python vol.py -d -f Win2008SP1x64-9de64630.vmss --profile=Win2008SP1x64
volshell
Current context: process System, pid=4, ppid=0 DTB=0x124000
Welcome to volshell! Current memory image is:
file:///Users/Michael/Downloads/Win2008SP1x64-9de64630.vmss
To get help, type 'hh()'
>>
self.addrspace.is_valid_address(0xfffffa8004a86a08L)
True
>> self.addrspace.vtop(0xfffffa8004a86a08L)
5354580488L
>> dd(0xfffffa8004a86a08L)
Memory
unreadable at fffffa8004a86a08
So the AS thinks the virtual address is valid and is able to vtop, but then
when you try to read (dd command) it fails. The first thing that catches my
eye is the physical address is reportedly 5354580488L, which is much larger
than the size of the file we have:
$ ls -al Win2008SP1x64-9de64630.vmss
-rw-r--r--@ 1 Michael staff 4300360567 Jul 6 16:01
/Users/Michael/Downloads/Win2008SP1x64-9de64630.vmss
This is the same thing we saw recently in the issue "vtop and 5GB 64bit
memory dump problem" [1]. That too was an issue of vmware memory files (a
vmem in this case unless Sebastien changed the extension). Same symptom,
but with the AMD64 AS - it reported vtop as being a physical address much
bigger than the file.
We're still looking into it some things, and although your AS could use a
little work to conform its style with the other AS's, I'm not sure its the
cause of the problem we're seeing here (unless the AMD64 AS has the same
problem).
Stay tuned.... ;-)
MHL
[1].
http://code.google.com/p/volatility/issues/detail?id=272
On Fri, Jul 6, 2012 at 5:58 PM, nir izraeli <nirizr(a)gmail.com> wrote:
> hi Michael,
> would you mind to also post your comments at the tracking system?
> it'll be a lot easier for me to keep track of it. hoping I'm not stepping
> on your tows.
> about the zread() - i didn't implement
it, I got confused with a few old
> AS classes that had unnecessary methods and probably also removed the
> zread() by mistake.
> I hope to fix these in a couple of days and
resubmit an updated version.
> the only issue i have trouble with is the
conversion to the internal
> object system.
> I tried using it a couple of times but had trouble with it.
> it would also duplicate efforts for writing modifications to the vmss
> parsing code.
> since it does seem to be easy to write structures using Volatility's
> framework,
> would you mind taking care of it yourselves?
> I could add a textual documentation if you'd rather, since i'll write one
> anyway.
> although if it's important i could give it another try...
> Thanks,
> Nir
> On Fri, Jul 6, 2012 at 11:49 PM, Michael Cohen <scudette(a)gmail.com>
wrote:
>> It looks to me like the address space
is not implementing zread()
>> properly (or at all). Can you please make sure that you are
>> implementing zread() in such a way that when you read outside a valid
>> or mapped region you will receive a null padded buffer rather than
>> None?
>
>> Some more comments about the address
space VMWareSnapshotFile:
>> - Please do not use inner classes. There is no need to have a class
>> defined in such a way - just place the class at the module level.
>> - Minor style issues - long lines should be wrapped at 80 chars,
>> commented out lines should be removed.
>> - Do no use double underscore member variable names (they mean
>> something specific e.g. self.__hasseek).
>> - It would also be nicer if we used the volatility object system
>> rather than struct module directly for parsing these things - it would
>> make the file formats more readable and simplify the code a lot.
>
>> Thanks
>> Michael.
>
>> On 6 July 2012 16:03, Jesse Bowling
<jessebowling(a)gmail.com> wrote:
>> > Disclaimer:
>>
>> > So I
took Nir's files, and dropped them into my plugins folder...I did
>> not
>> > see any new plugins using vol.py -h, and when I tried to do an
>> imageinfo I
>> > got:
>>
>> >
/usr/local/src/volatility-read-only-may-01/vol.py -f myimage.vmss
>> imageinfo
>>
>> >
Volatile Systems Volatility Framework 2.1_alpha
>> > Determining profile based on KDBG search...
>>
>> >
Traceback (most recent call last):
>> > File "/usr/local/src/volatility-read-only-may-01/vol.py", line
173, in
>> > <module
>> >
main()
>> > File "/usr/local/src/volatility-read-only-may-01/vol.py", line
164, in
>> > main
>> > command.execute()
>> > File
>> "/usr/local/src/volatility-read-only-may-01/volatility/commands.py",
>> > line 101, in execute
>> > func(outfd, data)
>> > File
>>
>>
"/usr/local/src/volatility-read-only-may-01/volatility/plugins/imageinfo.py",
>> > line 34, in render_text
>> > for k, v in data:
>> > File
>>
>>
"/usr/local/src/volatility-read-only-may-01/volatility/plugins/imageinfo.py",
>> > line 44, in calculate
>> > suglist = [ s for s, _, _ in kdbg.KDBGScan.calculate(self)]
>> > File
>>
>>
"/usr/local/src/volatility-read-only-may-01/volatility/plugins/kdbgscan.py",
>> > line 119, in calculate
>> > for offset in scanner.scan(aspace):
>> > File
>>
>>
"/usr/local/src/volatility-read-only-may-01/volatility/plugins/kdbgscan.py",
>> > line 83, in scan
>> > for offset in scan.BaseScanner.scan(self, address_space, offset,
>> > maxlen):
>> > File
"/usr/local/src/volatility-read-only-may-01/volatility/scan.py",
>> line
>> > 136, in scan
>> > skip = max(skip, s.skip(data, i))
>> > File
>>
>>
"/usr/local/src/volatility-read-only-may-01/volatility/plugins/common.py",
>> > line 49, in skip
>> > nextval = data.index(self.tag, offset + 1)
>> > AttributeError: 'NoneType' object has no attribute 'index'
>>
>> > So:
>>
>> > #
/usr/local/src/volatility-read-only-may-01/vol.py -f myimage.vmss
>> psscan
>>
>> >
Volatile Systems Volatility Framework 2.1_alpha
>> > Offset(P) Name PID PPID PDB Time created
>> > Time exited
>> > ---------- ---------------- ------ ------ ----------
>> > ------------------------ ------------------------
>> > No suitable address space mapping found
>> > Tried to open image as:
>> > WindowsHiberFileSpace32: No base Address Space
>> > VMWareSnapshotFile: No base Address Space
>> > WindowsCrashDumpSpace32: No base Address Space
>> > AMD64PagedMemory: No base Address Space
>> > JKIA32PagedMemory: No base Address Space
>> > JKIA32PagedMemoryPae: No base Address Space
>> > IA32PagedMemoryPae: Module disabled
>> > IA32PagedMemory: Module disabled
>> > WindowsHiberFileSpace32: No xpress signature found
>> > WindowsHiberFileSpace32: No xpress signature found
>> > VMWareSnapshotFile: ('Header signature invalid', 4026597203)
>> > WindowsCrashDumpSpace32: Header signature invalid
>> > AMD64PagedMemory: Incompatible profile WinXPSP2x86 selected
>> > JKIA32PagedMemory: Failed valid Address Space check
>> > JKIA32PagedMemoryPae: Failed valid Address Space check
>> > IA32PagedMemoryPae: Module disabled
>> > IA32PagedMemory: Module disabled
>> > FileAddressSpace: Must be first Address Space
>>
>> > At
least it doesn't crash. So now:
>>
>> > #
/usr/local/src/volatility-read-only-may-01/vol.py -f myimage.vmss
>> > --profile=Win2008R2SP1x64 psscan
>>
>> >
Volatile Systems Volatility Framework 2.1_alpha
>> > Offset(P) Name PID PPID PDB Time created
>> > Time exited
>> > ---------- ---------------- ------ ------ ----------
>> > ------------------------ ------------------------
>> > Traceback (most recent call last):
>> > File "/usr/local/src/volatility-read-only-may-01/vol.py", line
173, in
>> > <module
>> >
main()
>> > File "/usr/local/src/volatility-read-only-may-01/vol.py", line
164, in
>> > main
>> > command.execute()
>> > File
>> "/usr/local/src/volatility-read-only-may-01/volatility/commands.py",
>> > line 101, in execute
>> > func(outfd, data)
>> > File
>>
>>
"/usr/local/src/volatility-read-only-may-01/volatility/plugins/filescan.py",
>> > line 415, in render_text
>> > for eprocess in data:
>> > File
>>
>>
"/usr/local/src/volatility-read-only-may-01/volatility/plugins/filescan.py",
>> > line 405, in calculate
>> > for offset in PoolScanProcess().scan(address_space):
>> > File
"/usr/local/src/volatility-read-only-may-01/volatility/scan.py",
>> line
>> > 218, in scan
>> > for i in BaseScanner.scan(self, address_space, offset, maxlen):
>> > File
"/usr/local/src/volatility-read-only-may-01/volatility/scan.py",
>> line
>> > 136, in scan
>> > skip = max(skip, s.skip(data, i))
>> > File
>>
>>
"/usr/local/src/volatility-read-only-may-01/volatility/plugins/common.py",
>> > line 49, in skip
>> > nextval = data.index(self.tag, offset + 1)
>> > AttributeError: 'NoneType' object has no attribute 'index'
>>
>> > #
/usr/local/src/volatility-read-only-may-01/vol.py -f myimage.vmss
>> > --profile=Win2008R2SP1x64 --dtb=0x187000 psscan
>>
>> >
Volatile Systems Volatility Framework 2.1_alpha
>> > Offset(P) Name PID PPID PDB Time created
>> > Time exited
>> > ---------- ---------------- ------ ------ ----------
>> > ------------------------ ------------------------
>> > Traceback (most recent call last):
>> > File "/usr/local/src/volatility-read-only-may-01/vol.py", line
173, in
>> > <module
>> >
main()
>> > File "/usr/local/src/volatility-read-only-may-01/vol.py", line
164, in
>> > main
>> > command.execute()
>> > File
>> "/usr/local/src/volatility-read-only-may-01/volatility/commands.py",
>> > line 101, in execute
>> > func(outfd, data)
>> > File
>>
>>
"/usr/local/src/volatility-read-only-may-01/volatility/plugins/filescan.py",
>> > line 415, in render_text
>> > for eprocess in data:
>> > File
>>
>>
"/usr/local/src/volatility-read-only-may-01/volatility/plugins/filescan.py",
>> > line 405, in calculate
>> > for offset in PoolScanProcess().scan(address_space):
>> > File
"/usr/local/src/volatility-read-only-may-01/volatility/scan.py",
>> line
>> > 218, in scan
>> > for i in BaseScanner.scan(self, address_space, offset, maxlen):
>> > File
"/usr/local/src/volatility-read-only-may-01/volatility/scan.py",
>> line
>> > 136, in scan
>> > skip = max(skip, s.skip(data, i))
>> > File
>>
>>
"/usr/local/src/volatility-read-only-may-01/volatility/plugins/common.py",
>> > line 49, in skip
>> > nextval = data.index(self.tag, offset + 1)
>> > AttributeError: 'NoneType' object has no attribute 'index'
>>
>> > I have
limited testing time the next couple weeks, so will look to see
>> if I
>> > can share this with someone like SA in the meantime...
>>
>> >
Cheers,
>>
>> > Jesse
>>
>>
>> > On Fri, Jul 6, 2012 at 7:21 AM,
nir izraeli <nirizr(a)gmail.com> wrote:
>> >
>> >> I
assume you need it for something other than test my patch,
>> >> I can send parts of the vmss of the machine I already noticed more than
>> >> one region.
>> >> could you use that to gather the info you need?
>> >
>> >>
btw, I'm also using vmware converter standalone pretty often, it might
>> >> also be related
>> >
>> >
>> >> On Fri, Jul 6, 2012 at 5:31
AM, AAron Walters <awalters(a)4tphi.net
>> wrote:
>> >>
>> >>
>> >>> Nir,
>> >>
>> >>
>> >>>> AAron - actually it
was quite rare, but the first vmss I used to test
>> >>>> the patch
>> >>>> had two or three, which made my patch break when i first tested
it on
>> >>>> other
>> >>>> VMs.
>> >>>> I could try to pinpoint it, but i guess it would be easier for
me to
>> >>>> reverse
>> >>>> the vmware code than try it manually :)
>> >>>> A thing to note is that that vmss also had two virtual CPUs,
which
>> might
>> >>>> have
>> >>>> caused having more than one region. it also had ~4G of RAM. most
of
>> the
>> >>>> other
>> >>>> VMs i used only had about 512M.
>> >>>> did you try to run it on other vmss files that resemble the one
i
>> >>>> described?
>> >>
>> >>
>> >>> Interesting. I have never
seen a vmss with multiple regions. If you
>> >>> happen to come across one again, please let me know. I'd be
>> interested in
>> >>> what conditions or what product leads to more than one region.
>> >>
>> >>>
Thanks,
>> >>
>> >>>
AW
>> >
>> >
>>
>>
>>
>> > --
>> > Jesse Bowling
>>
>>
>>
>> > _______________________________________________
>> > Vol-users mailing list
>> > Vol-users(a)volatilityfoundation.org
>> >
http://lists.volatilityfoundation.org/mailman/listinfo/vol-users
>>
>
>
_______________________________________________
> Vol-users mailing list
> Vol-users(a)volatilityfoundation.org
>
http://lists.volatilityfoundation.org/mailman/listinfo/vol-users