September 2009 - Vol-dev - volatilityfoundation.org

by Michael Felber , Steufa Chemnitz, IT-Forensik

Hello Jesse, hello list, today I have given a try to the cryptoscan-plugin. The dump comes from an XP with SP3. That should not be problematic because the structure the plugin looks for is os-independent, isn't it? In the case I forced Truecrypt (v6.2a) to cache the passphrases in memory I saw it as plain text: XWF was not able to allocate that offset (phys. 0x18218c84) to a single process. But I was not able to find the described structure neither with the plugin nor manually. The dump is from a test case I use for forensic classes. So I could provide it for further analysis. It additionally includes cached domain credentials, waiting for extraction.. Cu Michael

15 years

2
1
0 0

Re: Pylint and code questions

by Michael Felber

Hello Mike, thanks a lot for the efforts you spent on that code optimization. I am a programming noob (I am a forensics guy) but will look at it too to learn. The firewire-access is quite funny but doesn't work with Vista at the moment (the raw1394 is unable to see the Vista PC). I have applied an extension (idea found at https://www.moonloop.org/bin/view/Moonloop/Stream?tag=ida) to winlockpwn so it works fine with any uptodate SP3 system I have testet. But a live analysis of the memory may be not the best idea because its to volatile, even a memory dump is not as consistent as a hiberfil.sys or memory.dmp would be. I would suggest making a memory dump first before analysis, except code injection techniques. ;-) Btw. Do you know how to extract the LSA-key from a SP3 registry, see cachedump.py. Jon, F-Response is a real great tool, you're right. It uses the iSCSI-protocol for communication and silently drops every write attempt. Both ways (FW / iSCSI) give the investigator the unlimited remote access to the 'living' memory of the target. F-response prevents writing but needs it own memory space, firewire does create a new device on the target system but does not consume memory. The main danger for changing evidences is its ability to write to the target. Cu Mic -- Jetzt kostenlos herunterladen: Internet Explorer 8 und Mozilla Firefox 3 - sicherer, schneller und einfacher! http://portal.gmx.net/de/go/chbrowser

15 years

4
5
0 0

Pylint and code questions

by Mike Auty

Hiya guys, First off thanks for the warm reception, everyone's been really friendly on IRC, etc. It feels like a really nice community. 5:) People on IRC suggest that the mailing list really is the right place for patches (even code clean-up patches), so I've unleashed my giant pylint patch on it. As I mentioned I've run pylint against most it (I use pydev with eclipse, which has pylint support built in, and went through all the files). Unfortunately that led to a horribly large 6000+ line patch (attached) and I'm really, really sorry it's such an unweildy nasty patch. 5:( It applies to the current volatility HEAD (r16), but it has a *lot* of changes it in, so I'd really appreciate it if someone could look over it (with me, if that helps). Mostly they should be indentation, spacing, unused imports and unused variables, but it could be that the change to explicit imports or prepending unused variable names with an _ might have an unexpected effect. I don't have that many memory images to run this against, so anyone who has more do please let me know if there's any problems. It might be easiest if I cut this down into file-by-file patches (although scan.py and vmodules.py were both huge). Would that be useful, or are people happy to deal with it as is? I did spot a couple of issues as I was going through, there's a method without the self argument in addrspace, and a few others which I was able to fix myself. Some of them though I don't know the code well enough to sort myself and I've listed here: * In scan.py, check_connection_poolprevioussize, seems always to return False, and sets PreviousSize seemingly for no reason, does the function have side effects besides simply returning? * In scan2.py, BaseMemoryScanner.Process uses a default value of {} which is generally a bad idea (having a mutable object as a default means that it may not always be what you expect when you call the function[1]). This turned up in a few other places later on as well, but I think I've fixed those up. * In tasks.py, find_dtb appears to reopen an existing address space rather than using the one it was provided with, and I can't quite figure out why? * In object.py, get_obj_offset, there's a variable that could potentially be used before it's defined (I think it's conditional and gets defined lower in the loop, so as long as the condition is never met first time round, everything will work fine). Is there a coding standard or anything that you guys are working to? So, yeah, I'm not sure the best way of approaching all this, but I thought I'd best post it to get it out there. I've also got a fair few questions about some bits and pieces I'd like to work on (with varying levels of viability) include: * Removing the big chunks of data table sat in crash_dump.py (I've already slimmed them down in another patch that'd need applying after the pylint patch, but I'm thinking of loading it all from a resource file or something). * I'd also like to remove the use of globals. I haven't figured out exactly where this gets used (it seems mostly to be various tables in meta_info.py), but I think it's generally discouraged. * It felt like there was an awful lot of repetition going on in vmodules.py, and I wondered if the functions couldn't be turned into classes and dumped in an internal-plugins location, to save hardwiring in everything in to "modules" at the start of the volatility script. * Also, the namespace seemed a bit all over the place, format_time was defined in two places, and most of the scripts seemed to just "from forensics.win32.blah import *", which imported all the imports, so lots of them were getting gmtime and strftime from modules other than time. Unfortunately I'm not around much this weekend, so if there's any issues or fixes or anything, they'll have to wait until Sunday, but hopefully I'll chat to people on the list or IRC. Thanks, Mike 5:) [1] http://effbot.org/zone/default-values.htm

15 years

2
1
0 0

Plugin input mechanism

by Mike Auty

Hiya guys, I'm pretty new here, so forgive me for bursting in and asking lots of questions, but I'm trying to get used to the codebase (I've had some fun running pylint across this, which helped a *lot* with trying to figure out what went where). I'd really like to add a layer that allows the existing volatility tools to be run against a live system, directly via firewire (ie, without having to completely image the system first). I should probably mention now that I'm from a penetration testing background, rather than a forensics background, but that I've been investigating the firewire technique metlstorm developed for a while now. My hope is that some of the hash dumping tools and similar might be much quicker when run directly, rather than having to image the entire system first, and that it might also help the volatility suite be used as a live debugging aide for people trying to learn about memory and how it works. It seemed as though everything was laid out nicely, so I'd just need to implement a FirewireAddressSpace object (which I've done, and attached, although I have yet to test it). The problem I found was that all forensics.commands.command subclasses appear to accept an optparse parser result, and then manually open their own image based on the filename. Since most of the functions seem to operate on an AddressSpace rather than on a filename string, it seemed as though it should be possible to pass that into the plugins/commands. So that leads me to a few questions: * Would it be worthwhile reworking the module system to allow modules to specify their requirements more abstractly (I need a memory image, I need a set of registry keys, etc)? * Would it need to be that abstract, or do pretty much all modules always act on an AddressSpace? * How are you guys intending to deal with upgrading plugins after a major enhancement (I noticed plugins are already getting a bit out of hand, since Andreas' plugins require overwriting bits of volatility with his own versions)? * Would it be a major version number bump with an API break, and if so are there any other changes that would be worthwhile addressing at the same time? * Should I investigate allowing multiple plugin versions to co-exist? In implementing a new AddressSpace, I also spotted this fast variable, that appears simply to open a new handle, and I can't figure out how that helps or makes it faster? Was this a place holder for a possible address space enhancement, or is there really some kind of speed-up which I'm missing? Sorry to kind of drop all this here, but it seemed the most appropriate place for it. I've joined the IRC channel as well, so if you want to poke me for not doing things sensibly, you'll find me on there as ikelos... 5:) Thanks, Mike 5:) diff --git a/Volatility/forensics/addrspace.py b/Volatility/forensics/addrspace.py index d0c0dee..8697f71 100755 --- a/Volatility/forensics/addrspace.py +++ b/Volatility/forensics/addrspace.py @@ -30,6 +30,7 @@ """ Alias for all address spaces """ import os +import sys import struct try: @@ -73,7 +74,7 @@ class FileAddressSpace: return False return addr < self.fsize - 1 - def close(): + def close(self): self.fhandle.close() class BufferAddressSpace(FileAddressSpace): @@ -85,3 +86,103 @@ class BufferAddressSpace(FileAddressSpace): if fast == True: self.fast_fhandle = StringIO(buf) + +try: + import firewire +except: + pass + +if 'firewire' in sys.modules: + + def intervals(exclusions, start, end): + """Returns a list of intervals, from start to end, that do not include the exclusions""" + return _intervals(sorted(exclusions), start, end, []) + + def _intervals(exclusions, start, end, accumulator): + """Accepts a sorted list of intervals and a start and end + + This will return a list of intervals between start and length + that does not contain any of the intervals in the list of exclusions. + """ + if not len(exclusions): + # We're done + return accumulator + [(start, end)] + + e = exclusions[0] + estart = e[0] + eend = e[1] + + # e and range overlap + if not (eend < start or estart > end): + # Ignore this exclusions + return intervals(exclusions[1:], start, end, accumulator) + if estart < start: + if eend < end: + # Covers the start of the remaining length + return intervals(exclusions[1:], eend, end, accumulator) + else: + # Covers the entire remaining area + return accumulator + else: + if eend < end: + # Covers a section of the remaining length + return intervals(exclusions[1:], eend, end, accumulator + [(start, estart)]) + else: + # Covers the end of the remaining length + return accumulator + [(start, estart)] + + class FirewireIO(object): + """File-like interface for Firewire devices""" + def __init__(self, bus, node): + self.offset = 0 + self.fsize = 4 * 1024 * 1024 * 1024 + self.closed = False + self._exclusions = [(0xa0000, 0xfffff, "Upper Memory Area")] + try: + h = firewire.Host() + self._node = h[bus][node] + except IndexError: + raise IOError, "Firewire node " + str(node) + " on bus " + str(bus) + " was not accessible" + + def close(self): + """Close function for file-like interface""" + self.closed = True + + def flush(self): + """Flush function for file-like interface""" + pass + + def read(self, size = -1): + if size < 0: + size = self.fsize + ints = intervals(self._exclusions, self.offset, self.offset + size) + output = "\x00" * size + for i in ints: + output = output[: i[0] - self.offset] + self._node.read(i[1] - i[0], i[0]) + output[i[1] - self.offset:] + return output + + def seek(self, offset, whence = os.SEEK_SET): + """Seek function for file-like interface""" + self.offset = offset + if (whence == os.SEEK_CUR): + self.offset += offset + elif (whence == os.SEEK_END): + self.offset = self.fsize - offset + + def tell(self): + """Tell function for file-like interface""" + return self.offset + + def write(self, string): + pass + + class FirewireAddressSpace(FileAddressSpace): + def __init__(self, bus, node, fast=False): + self.fname = "Firewire on bus " + bus + " node " + node + self.name = self.fname + self.fhandle = FirewireIO(bus, node) + # Theoretical limit + self.fsize = self.fhandle.fsize + + if fast == True: + raise RuntimeError("Fast mode for firewire not implemented") \ No newline at end of file

15 years

1
0
0 0

AW: CashDump-Issue

by Michael Felber , Steufa Chemnitz, IT-Forensik

Oh, oh, forget my last posting: I've used the wrong offset for the SECURITY-Hive within that example. SORRY. Using an older SP2-dump it worked fine. Found no wrong code within the script. ;-) But the script was not able to extract the LSA key from a SP3-dump. Does anyone have a solution for that? How the LSA key may get extracted otherwise? It MUST be somewhere in the lsass-process, right? TNX in advance and sorry for the wrong posting again. Cu Michael Von: Michael Felber , Steufa Chemnitz, IT-Forensik [mailto:MichaelFelber@gmx.net] Gesendet: Donnerstag, 10. September 2009 16:42 An: 'vol-dev(a)volatilityfoundation.org' Betreff: CashDump-Issue Hello, according a hint of Andreas (TNX!!) I've tackled the problem of extracting cached domain credentials from a memory-dump. At the end of my path of epiphany I saw that Volatility already has a plugin doing that: hashdump.py. Great. While giving it a try I only got error messages like Traceback (most recent call last): File "volatility", line 219, in <module> main() File "volatility", line 215, in main command.execute() File "memory_plugins\registry/hashdump.py", line 78, in execute dump_memory_hashes(addr_space, types, self.opts.syshive, self.opts.samhive, prof) File "C:\Micha\Forensics\Volatility\forensics\win32\hashdump.py", line 305, in dump_memory_hashes dump_hashes(sysaddr, samaddr, profile) File "C:\Micha\Forensics\Volatility\forensics\win32\hashdump.py", line 289, in dump_hashes bootkey = get_bootkey(sysaddr,profile) File "C:\Micha\Forensics\Volatility\forensics\win32\hashdump.py", line 131, in get_bootkey class_data = sysaddr.read(key.Class, key.ClassLength) AttributeError: 'NoneType' object has no attribute 'Class' >From my point of view as a programming noob some type of type declaration is missed. Or did I miss something? I have applied all the recent patches posted in this list. The full console dump is attached for kindly being reviewed. Cu Mic

15 years

1
0
0 0

CashDump-Issue

by Michael Felber , Steufa Chemnitz, IT-Forensik

Hello, according a hint of Andreas (TNX!!) I've tackled the problem of extracting cached domain credentials from a memory-dump. At the end of my path of epiphany I saw that Volatility already has a plugin doing that: hashdump.py. Great. While giving it a try I only got error messages like Traceback (most recent call last): File "volatility", line 219, in <module> main() File "volatility", line 215, in main command.execute() File "memory_plugins\registry/hashdump.py", line 78, in execute dump_memory_hashes(addr_space, types, self.opts.syshive, self.opts.samhive, prof) File "C:\Micha\Forensics\Volatility\forensics\win32\hashdump.py", line 305, in dump_memory_hashes dump_hashes(sysaddr, samaddr, profile) File "C:\Micha\Forensics\Volatility\forensics\win32\hashdump.py", line 289, in dump_hashes bootkey = get_bootkey(sysaddr,profile) File "C:\Micha\Forensics\Volatility\forensics\win32\hashdump.py", line 131, in get_bootkey class_data = sysaddr.read(key.Class, key.ClassLength) AttributeError: 'NoneType' object has no attribute 'Class' >From my point of view as a programming noob some type of type declaration is missed. Or did I miss something? I have applied all the recent patches posted in this list. The full console dump is attached for kindly being reviewed. Cu Mic

15 years

1
0
0 0

Re: Patch: strings command optimization

by Jamie Levy

Kind of diverging from the topic at hand, but: > sed 's/ /:/' > > to do the conversion myself. (Note that sed will only perform one > replacement per line by default.) You can do global replacement like so: sed 's/ /:/g' All the best, -Jamie

15 years

3
3
0 0

Patch: strings command optimization

by Tim

Hello, Please find attached a patch to speed up the Volatility strings command. On my slow laptop, a particular search for a single string required around 2 hours and 25 minutes. After these changes, the same search requires 35 minutes. (Measured in wall clock time.) Notable changes include: - Switched from guess-and-check style address probing (during the reverse map construction) to using get_available_pages(). On my test image, this cuts the number of vtop() calls down from 1M to something on the order of 55K for each process. This more than made up for the additional overhead of calling get_available_pages(). - The above change also eliminates a 32bit address space assumption appearing in top-level code, which I assume is a good thing. - Reduced the number of reverse_map lookups - Moved parsing of the strings input file to an earlier point in processing so users don't have to wait until the end of the job to discover they didn't specify things correctly. I can think of several other ways to make this faster and/or more memory efficient, but most require changes to the address interface (ever thought of providing a page iterator?). Also, I added a few comments for future improvements to the user interface and output. I think currently there are problems with the way that addresses appearing in large pages are reported, but I'm not 100% sure what the output is intended to mean, so I just kept it the same for now. DISCLAIMER: This is the first time I've seriously looked at the Volatility code. Please review/test it carefully before committing the changes. thanks! tim

15 years

3
3
0 0

latest

by zahra zohoor

Hi, what are the latest enhancements made to Volatility in Windows platform? Best regards Zahra

15 years

1
0
0 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

Vol-dev September 2009