Re: [Vol-users] Windows Server 2008

Hey Nir, We can definitely help out with integrating your code with the object system. I was just in the process of testing it for the first time this afternoon. Here are a few details (I'll copy them to the issue tracker in a sec). Basically I started with an ESX 4.1.0 and grabbed the following: * vmsn from xpsp2 x86 256 MB * vmsn from win7 sp1 x86 512 MB * vmss from server 2008 sp1 x64 4 GB Your AS with the latest 2.1 alpha (so about r1983) worked fine for the xp vmsn. (by fine I mean pslist worked, but other plugins may not work properly due to what scudette said about zread) ================================================= VMSN - XPSP3 x86 @ 256 MB RAM $ python vol.py -d -f Andrew-Snapshot6.vmsn pslist Volatile Systems Volatility Framework 2.1_alpha [snip] DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.vmware.VMWareSnapshotFile'> DEBUG : volatility.plugins.addrspaces.vmware: RegionCount: 1 DEBUG : volatility.plugins.addrspaces.vmware: Virtual Address: 0, Physical Address: 0, Size: 10000000 DEBUG : volatility.plugins.addrspaces.vmware: dtb: 39000 DEBUG : volatility.utils : Succeeded instantiating <volatility.plugins.addrspaces.vmware.VMWareSnapshotFile object at 0x10363a890> Offset(V) Name PID PPID Thds Hnds Sess Wow64 Start Exit ---------- -------------------- ------ ------ ------ -------- ------ ------ -------------------- -------------------- 0x81bcc830 System 4 0 52 477 ------ 0 0x8194a020 smss.exe 364 4 3 21 ------ 0 2012-01-25 20:44:20 0x81954020 csrss.exe 616 364 10 345 0 0 2012-01-25 20:44:20 0x81951128 winlogon.exe 640 364 16 495 0 0 2012-01-25 20:44:20 0x81a897a8 services.exe 684 640 15 272 0 0 2012-01-25 20:44:20 [snip] It also worked fine for the win7 vmsn: ================================================= VMSN - Windows 7 SP0 x86 @ 512 MB RAM $ python vol.py -d -f Abraham-Snapshot2.vmsn --profile=Win7SP0x86 pslist Volatile Systems Volatility Framework 2.1_alpha [snip] DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.vmware.VMWareSnapshotFile'> DEBUG : volatility.plugins.addrspaces.vmware: RegionCount: 1 DEBUG : volatility.plugins.addrspaces.vmware: Virtual Address: 0, Physical Address: 0, Size: 20000000 DEBUG : volatility.plugins.addrspaces.vmware: dtb: 185000 DEBUG : volatility.utils : Succeeded instantiating <volatility.plugins.addrspaces.vmware.VMWareSnapshotFile object at 0x102a1a750> Offset(V) Name PID PPID Thds Hnds Sess Wow64 Start Exit ---------- -------------------- ------ ------ ------ -------- ------ ------ -------------------- -------------------- 0x83f2f730 System 4 0 93 494 ------ 0 2012-03-15 15:04:12 0x84f32c48 smss.exe 252 4 2 29 ------ 0 2012-03-15 15:04:12 0x85708d40 csrss.exe 364 356 9 386 0 0 2012-03-15 15:04:48 0x82050030 wininit.exe 400 356 3 75 0 0 2012-03-15 15:04:48 0x856d9370 csrss.exe 408 392 7 201 1 0 2012-03-15 15:04:48 0x8207f030 services.exe 468 400 8 198 0 0 2012-03-15 15:04:49 0x8208e030 lsass.exe 476 400 8 711 0 0 2012-03-15 15:04:49 [snip] I then reproduced the same thing Jesse is seeing on the server 2008 x64 w/ 4 GB: ================================================= VMSS - Server 2008 SP1 x64 @ 4 GB RAM $ python vol.py -d -f Win2008SP1x64-9de64630.vmss --profile=Win2008SP1x64 pslist Volatile Systems Volatility Framework 2.1_alpha [snip] DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.hibernate.WindowsHiberFileSpace32'> DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.crash.WindowsCrashDumpSpace64'> DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.crash.WindowsCrashDumpSpace32'> DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.vmware.VMWareSnapshotFile'> DEBUG : volatility.plugins.addrspaces.vmware: Read region count from file: 2 DEBUG : volatility.plugins.addrspaces.vmware: RegionCount: 2 DEBUG : volatility.plugins.addrspaces.vmware: Virtual Address: 0, Physical Address: 0, Size: C0000000 Virtual Address: 100000000, Physical Address: C0000000, Size: 40000000 DEBUG : volatility.plugins.addrspaces.vmware: dtb: 124000 DEBUG : volatility.utils : Succeeded instantiating <volatility.plugins.addrspaces.vmware.VMWareSnapshotFile object at 0x102a1a750> DEBUG : volatility.utils : Voting round DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.hibernate.WindowsHiberFileSpace32'> DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.crash.WindowsCrashDumpSpace64'> DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.crash.WindowsCrashDumpSpace32'> DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.vmware.VMWareSnapshotFile'> DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.amd64.AMD64PagedMemory'> DEBUG : volatility.utils : Succeeded instantiating <volatility.plugins.addrspaces.amd64.AMD64PagedMemory object at 0x1036c91d0> DEBUG : volatility.utils : Voting round DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.hibernate.WindowsHiberFileSpace32'> DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.crash.WindowsCrashDumpSpace64'> DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.crash.WindowsCrashDumpSpace32'> DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.vmware.VMWareSnapshotFile'> DEBUG : volatility.utils : Failed instantiating (exception): unpack requires a string argument of length 4 DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.amd64.AMD64PagedMemory'> DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.intel.JKIA32PagedMemory'> DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.legacyintel.IA32PagedMemoryPae'> DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.intel.JKIA32PagedMemoryPae'> DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.legacyintel.IA32PagedMemory'> DEBUG : volatility.utils : Trying <class 'volatility.plugins.addrspaces.standard.FileAddressSpace'> Offset(V) Name PID PPID Thds Hnds Sess Wow64 Start Exit ------------------ -------------------- ------ ------ ------ -------- ------ ------ -------------------- -------------------- 0xfffffa8003ca8950 System 4 0 104 496 ------ 0 2012-03-02 07:16:23 /Users/Michael/volatility_pe_exceptions/volatility/plugins/overlays/windows/windows.py(262)windows_to_unix_time() -> unix_time = windows_time / 10000000 (Pdb) windows_time <NoneObject: Unable to read 8 bytes from 18446738026473744904> You can see the error occurred because windows_time at 0xfffffa8004a86a08L (hex value of the decimal offset above) could not be fetched from the vmss file. Since it appears the System process is able to be found, we should be able to break into a volshell (which uses the System process AS by default) and try some checks: $ python vol.py -d -f Win2008SP1x64-9de64630.vmss --profile=Win2008SP1x64 volshell Current context: process System, pid=4, ppid=0 DTB=0x124000 Welcome to volshell! Current memory image is: file:///Users/Michael/Downloads/Win2008SP1x64-9de64630.vmss To get help, type 'hh()' True 5354580488L Memory unreadable at fffffa8004a86a08 So the AS thinks the virtual address is valid and is able to vtop, but then when you try to read (dd command) it fails. The first thing that catches my eye is the physical address is reportedly 5354580488L, which is much larger than the size of the file we have: $ ls -al Win2008SP1x64-9de64630.vmss -rw-r--r--@ 1 Michael staff 4300360567 Jul 6 16:01 /Users/Michael/Downloads/Win2008SP1x64-9de64630.vmss This is the same thing we saw recently in the issue "vtop and 5GB 64bit memory dump problem" [1]. That too was an issue of vmware memory files (a vmem in this case unless Sebastien changed the extension). Same symptom, but with the AMD64 AS - it reported vtop as being a physical address much bigger than the file. We're still looking into it some things, and although your AS could use a little work to conform its style with the other AS's, I'm not sure its the cause of the problem we're seeing here (unless the AMD64 AS has the same problem). Stay tuned.... ;-) MHL [1]. http://code.google.com/p/volatility/issues/detail?id=272 On Fri, Jul 6, 2012 at 5:58 PM, nir izraeli &lt;nirizr(a)gmail.com&gt; wrote: > hi Michael, > would you mind to also post your comments at the tracking system? > it'll be a lot easier for me to keep track of it. hoping I'm not > stepping on your tows. > > about the zread() - i didn't implement it, I got confused with a few old > AS classes that had unnecessary methods and probably also removed the > zread() by mistake. > > I hope to fix these in a couple of days and resubmit an updated version. > > the only issue i have trouble with is the conversion to the internal > object system. > I tried using it a couple of times but had trouble with it. > it would also duplicate efforts for writing modifications to the vmss > parsing code. > since it does seem to be easy to write structures using Volatility's > framework, > would you mind taking care of it yourselves? > I could add a textual documentation if you'd rather, since i'll write > one anyway. > although if it's important i could give it another try... > > Thanks, > Nir > > > On Fri, Jul 6, 2012 at 11:49 PM, Michael Cohen &lt;scudette(a)gmail.com&gt;wrote: > >> It looks to me like the address space is not implementing zread() >> properly (or at all). Can you please make sure that you are >> implementing zread() in such a way that when you read outside a valid >> or mapped region you will receive a null padded buffer rather than >> None? >> >> Some more comments about the address space VMWareSnapshotFile: >> - Please do not use inner classes. There is no need to have a class >> defined in such a way - just place the class at the module level. >> - Minor style issues - long lines should be wrapped at 80 chars, >> commented out lines should be removed. >> - Do no use double underscore member variable names (they mean >> something specific e.g. self.__hasseek). >> - It would also be nicer if we used the volatility object system >> rather than struct module directly for parsing these things - it would >> make the file formats more readable and simplify the code a lot. >> >> Thanks >> Michael. >> >> On 6 July 2012 16:03, Jesse Bowling &lt;jessebowling(a)gmail.com&gt; wrote: >> > Disclaimer: >>>> > So I took Nir's files, and dropped them into my plugins folder...I >> did not >> > see any new plugins using vol.py -h, and when I tried to do an >> imageinfo I >> > got: >>>> > /usr/local/src/volatility-read-only-may-01/vol.py -f myimage.vmss >> imageinfo >>>> > Volatile Systems Volatility Framework 2.1_alpha >> > Determining profile based on KDBG search... >>>> > Traceback (most recent call last): >> > File "/usr/local/src/volatility-read-only-may-01/vol.py", line 173, >> in >> > <module> >> > main() >> > File "/usr/local/src/volatility-read-only-may-01/vol.py", line 164, >> in >> > main >> > command.execute() >> > File >> "/usr/local/src/volatility-read-only-may-01/volatility/commands.py", >> > line 101, in execute >> > func(outfd, data) >> > File >>>> "/usr/local/src/volatility-read-only-may-01/volatility/plugins/imageinfo.py", >> > line 34, in render_text >> > for k, v in data: >> > File >>>> "/usr/local/src/volatility-read-only-may-01/volatility/plugins/imageinfo.py", >> > line 44, in calculate >> > suglist = [ s for s, _, _ in kdbg.KDBGScan.calculate(self)] >> > File >>>> "/usr/local/src/volatility-read-only-may-01/volatility/plugins/kdbgscan.py", >> > line 119, in calculate >> > for offset in scanner.scan(aspace): >> > File >>>> "/usr/local/src/volatility-read-only-may-01/volatility/plugins/kdbgscan.py", >> > line 83, in scan >> > for offset in scan.BaseScanner.scan(self, address_space, offset, >> > maxlen): >> > File >> "/usr/local/src/volatility-read-only-may-01/volatility/scan.py", line >> > 136, in scan >> > skip = max(skip, s.skip(data, i)) >> > File >>>> "/usr/local/src/volatility-read-only-may-01/volatility/plugins/common.py", >> > line 49, in skip >> > nextval = data.index(self.tag, offset + 1) >> > AttributeError: 'NoneType' object has no attribute 'index' >>>> > So: >>>> > # /usr/local/src/volatility-read-only-may-01/vol.py -f myimage.vmss >> psscan >>>> > Volatile Systems Volatility Framework 2.1_alpha >> > Offset(P) Name PID PPID PDB Time created >> > Time exited >> > ---------- ---------------- ------ ------ ---------- >> > ------------------------ ------------------------ >> > No suitable address space mapping found >> > Tried to open image as: >> > WindowsHiberFileSpace32: No base Address Space >> > VMWareSnapshotFile: No base Address Space >> > WindowsCrashDumpSpace32: No base Address Space >> > AMD64PagedMemory: No base Address Space >> > JKIA32PagedMemory: No base Address Space >> > JKIA32PagedMemoryPae: No base Address Space >> > IA32PagedMemoryPae: Module disabled >> > IA32PagedMemory: Module disabled >> > WindowsHiberFileSpace32: No xpress signature found >> > WindowsHiberFileSpace32: No xpress signature found >> > VMWareSnapshotFile: ('Header signature invalid', 4026597203) >> > WindowsCrashDumpSpace32: Header signature invalid >> > AMD64PagedMemory: Incompatible profile WinXPSP2x86 selected >> > JKIA32PagedMemory: Failed valid Address Space check >> > JKIA32PagedMemoryPae: Failed valid Address Space check >> > IA32PagedMemoryPae: Module disabled >> > IA32PagedMemory: Module disabled >> > FileAddressSpace: Must be first Address Space >>>> > At least it doesn't crash. So now: >>>> > # /usr/local/src/volatility-read-only-may-01/vol.py -f myimage.vmss >> > --profile=Win2008R2SP1x64 psscan >>>> > Volatile Systems Volatility Framework 2.1_alpha >> > Offset(P) Name PID PPID PDB Time created >> > Time exited >> > ---------- ---------------- ------ ------ ---------- >> > ------------------------ ------------------------ >> > Traceback (most recent call last): >> > File "/usr/local/src/volatility-read-only-may-01/vol.py", line 173, >> in >> > <module> >> > main() >> > File "/usr/local/src/volatility-read-only-may-01/vol.py", line 164, >> in >> > main >> > command.execute() >> > File >> "/usr/local/src/volatility-read-only-may-01/volatility/commands.py", >> > line 101, in execute >> > func(outfd, data) >> > File >>>> "/usr/local/src/volatility-read-only-may-01/volatility/plugins/filescan.py", >> > line 415, in render_text >> > for eprocess in data: >> > File >>>> "/usr/local/src/volatility-read-only-may-01/volatility/plugins/filescan.py", >> > line 405, in calculate >> > for offset in PoolScanProcess().scan(address_space): >> > File >> "/usr/local/src/volatility-read-only-may-01/volatility/scan.py", line >> > 218, in scan >> > for i in BaseScanner.scan(self, address_space, offset, maxlen): >> > File >> "/usr/local/src/volatility-read-only-may-01/volatility/scan.py", line >> > 136, in scan >> > skip = max(skip, s.skip(data, i)) >> > File >>>> "/usr/local/src/volatility-read-only-may-01/volatility/plugins/common.py", >> > line 49, in skip >> > nextval = data.index(self.tag, offset + 1) >> > AttributeError: 'NoneType' object has no attribute 'index' >>>> > # /usr/local/src/volatility-read-only-may-01/vol.py -f myimage.vmss >> > --profile=Win2008R2SP1x64 --dtb=0x187000 psscan >>>> > Volatile Systems Volatility Framework 2.1_alpha >> > Offset(P) Name PID PPID PDB Time created >> > Time exited >> > ---------- ---------------- ------ ------ ---------- >> > ------------------------ ------------------------ >> > Traceback (most recent call last): >> > File "/usr/local/src/volatility-read-only-may-01/vol.py", line 173, >> in >> > <module> >> > main() >> > File "/usr/local/src/volatility-read-only-may-01/vol.py", line 164, >> in >> > main >> > command.execute() >> > File >> "/usr/local/src/volatility-read-only-may-01/volatility/commands.py", >> > line 101, in execute >> > func(outfd, data) >> > File >>>> "/usr/local/src/volatility-read-only-may-01/volatility/plugins/filescan.py", >> > line 415, in render_text >> > for eprocess in data: >> > File >>>> "/usr/local/src/volatility-read-only-may-01/volatility/plugins/filescan.py", >> > line 405, in calculate >> > for offset in PoolScanProcess().scan(address_space): >> > File >> "/usr/local/src/volatility-read-only-may-01/volatility/scan.py", line >> > 218, in scan >> > for i in BaseScanner.scan(self, address_space, offset, maxlen): >> > File >> "/usr/local/src/volatility-read-only-may-01/volatility/scan.py", line >> > 136, in scan >> > skip = max(skip, s.skip(data, i)) >> > File >>>> "/usr/local/src/volatility-read-only-may-01/volatility/plugins/common.py", >> > line 49, in skip >> > nextval = data.index(self.tag, offset + 1) >> > AttributeError: 'NoneType' object has no attribute 'index' >>>> > I have limited testing time the next couple weeks, so will look to >> see if I >> > can share this with someone like SA in the meantime... >>>> > Cheers, >>>> > Jesse >>>>>> > On Fri, Jul 6, 2012 at 7:21 AM, nir izraeli &lt;nirizr(a)gmail.com&gt; wrote: >> >> >> >> I assume you need it for something other than test my patch, >> >> I can send parts of the vmss of the machine I already noticed more >> than >> >> one region. >> >> could you use that to gather the info you need? >> >> >> >> btw, I'm also using vmware converter standalone pretty often, it >> might >> >> also be related >> >> >> >> >> >> On Fri, Jul 6, 2012 at 5:31 AM, AAron Walters &lt;awalters(a)4tphi.net&gt; >> wrote: >> >>> >> >>> >> >>> Nir, >> >>> >> >>> >> >>>> AAron - actually it was quite rare, but the first vmss I used to >> test >> >>>> the patch >> >>>> had two or three, which made my patch break when i first tested it >> on >> >>>> other >> >>>> VMs. >> >>>> I could try to pinpoint it, but i guess it would be easier for me >> to >> >>>> reverse >> >>>> the vmware code than try it manually :) >> >>>> A thing to note is that that vmss also had two virtual CPUs, which >> might >> >>>> have >> >>>> caused having more than one region. it also had ~4G of RAM. most >> of the >> >>>> other >> >>>> VMs i used only had about 512M. >> >>>> did you try to run it on other vmss files that resemble the one i >> >>>> described? >> >>> >> >>> >> >>> Interesting. I have never seen a vmss with multiple regions. If you >> >>> happen to come across one again, please let me know. I'd be >> interested in >> >>> what conditions or what product leads to more than one region. >> >>> >> >>> Thanks, >> >>> >> >>> AW >> >> >> >> >>>>>>>> > -- >> > Jesse Bowling >>>>>>>> > _______________________________________________ >> > Vol-users mailing list >> > Vol-users(a)volatilityfoundation.org >> > http://lists.volatilityfoundation.org/mailman/listinfo/vol-users >>>> > > > _______________________________________________ > Vol-users mailing list > Vol-users(a)volatilityfoundation.org > http://lists.volatilityfoundation.org/mailman/listinfo/vol-users > >