12:59 p.m.
Hello,
Please find attached a patch to speed up the Volatility strings
command. On my slow laptop, a particular search for a single string
required around 2 hours and 25 minutes. After these changes, the same
search requires 35 minutes. (Measured in wall clock time.)
Notable changes include:
- Switched from guess-and-check style address probing (during the
reverse map construction) to using get_available_pages(). On my
test image, this cuts the number of vtop() calls down from 1M to
something on the order of 55K for each process. This more than made
up for the additional overhead of calling get_available_pages().
- The above change also eliminates a 32bit address space assumption
appearing in top-level code, which I assume is a good thing.
- Reduced the number of reverse_map lookups
- Moved parsing of the strings input file to an earlier point in
processing so users don't have to wait until the end of the job to
discover they didn't specify things correctly.
I can think of several other ways to make this faster and/or more
memory efficient, but most require changes to the address interface
(ever thought of providing a page iterator?).
Also, I added a few comments for future improvements to the user
interface and output. I think currently there are problems with the
way that addresses appearing in large pages are reported, but I'm not
100% sure what the output is intended to mean, so I just kept it the
same for now.
DISCLAIMER: This is the first time I've seriously looked at the
Volatility code. Please review/test it carefully before committing
the changes.
thanks!
tim
4:30 p.m.
Hi Tim,
I'm still testing your patches, but have found the following Perl
script necessary to convert my Mac's strings output into the format
needed by Volatility. I thought others might be able to make use of it:
#!/usr/bin/perl -w
while (<>)
{
$_ =~ s/^(\s*[0-9]+) /$1\:/;
print $_;
}
Usage:
$ strings -o [memoryimage] > raw
$ ./colon.pl < raw > strings.txt
$ python volatility strings -f [memoryimage] -s strings.txt
--
Jesse Kornblum
5:46 p.m.
Jesse,
I'm still testing your patches,
Cool, thanks for taking a look at it.
but have found the following Perl
script necessary to convert my Mac's strings output into the format
needed by Volatility. I thought others might be able to make use of
it:
[...]
Yeah, I've also been thinking that the strings command should be
changed to accept more standard strings output (i.e. a space as
delimiter). The change would be easy and I don't think it will break
anything, so long as it also accepts ':' as a delimiter. For now
I've just been using something like:
sed 's/ /:/'
to do the conversion myself. (Note that sed will only perform one
replacement per line by default.)
Once it is decided which way this command should be updated (i.e.
changed to a core plugin or something similar), I can come back and
make some more functional changes like this if there's no objections.
cheers,
tim
6:04 p.m.
As long as we're chatting about strings, it's worth noting that the
"strings" command on UNIX-like operating systems will not find UTF-16
encoded strings by default (like those commonly found in Windows).
If you've got a copy of GNU strings, you can use:
strings -t d -e l <file>
(the "-e l" tells it to look for strings in UTF-16 little-endian
encoding. Other encodings are possible too, check the manpage).
If you are unlucky enough to be without a copy of GNU strings (OS X
doesn't ship with it by default; it instead ships a non-unicode-aware
version), you can install it as part of binutils.
-Brendan
On Sep 7, 2009, at 5:46 PM, Tim wrote:
Jesse,
I'm still testing your patches,
Cool, thanks for taking a look at it.
but have found the following Perl
script necessary to convert my Mac's strings output into the format
needed by Volatility. I thought others might be able to make use of
it:
[...]
Yeah, I've also been thinking that the strings command should be
changed to accept more standard strings output (i.e. a space as
delimiter). The change would be easy and I don't think it will break
anything, so long as it also accepts ':' as a delimiter. For now
I've just been using something like:
sed 's/ /:/'
to do the conversion myself. (Note that sed will only perform one
replacement per line by default.)
Once it is decided which way this command should be updated (i.e.
changed to a core plugin or something similar), I can come back and
make some more functional changes like this if there's no objections.
cheers,
tim
_______________________________________________
Vol-dev mailing list
Vol-dev(a)volatilityfoundation.org
http://lists.volatilityfoundation.org/mailman/listinfo/vol-dev
5500
days inactive
5513
days old
vol-dev@lists.volatilityfoundation.org
3 comments
3 participants
participants (3)
-
Brendan Dolan-Gavitt -
Jesse Kornblum -
Tim