Michael Cohen wrote:
Hi Mike,
Hiya Michael,
Thanks for that - the patch didnt seem to make it for
me (it ended up
being a 14 byte bz2 file - maybe gmail stole it?).
You're quite right, the patch I attached was entirely and completely
non-existant (it was a bzip2 of an empty file). Sorry about that
everybody! 5:( You should find the full thing attached now.
Not exactly - the new object scheme itself doesnt need
these as these
functions were merged into the new object code. However, since I still
had the old object code lying around there might have been some
references left over.
There were, I think it's just the one call to list_do in
forensics.linux.tasks.
utils.py is in the svn tree (or it should be).
It's not (at least, not the volatility svn tree), it's included in this
giant patch though (as, unfortunately, is linked_list). The only other
thing that the code seemed to be missing before it would work correctly
was a definition of _LDR_DATA_TABLE_ENTRY, which this patch also adds.
Some of that work is already done - please have a look
at the new
plugins which are doing much the same things - I just left vmodules
there as a guide as to whats left to do. For example get_pslist() is
now powered by a rewritten pslist() in ./forensics/win32/tasks.py and
is used in the module memory_plugins/pstree.py for example.
I spotted pstree, it's the plugin I was going to use as the basis for
modifying the others to using load_as! 5:) Unfortunately, I've no idea
which plugins duplicate functionality, and which simply do things a
different way (all of the blahscan2 functions appear *much* quicker than
the blahscan functions, but produce pretty much the same output), so I
was going to convert them all and then ask people to help me cut away
the ones we don't need...
I dont think backwards compatibility is a realistic
goal we can have
when we change the underlying framework. I am more happy with a linux
style development model where everyone is free to have their own
modules, but committing them to the main tree will ensure they get
maintained as the code evolves.
That sounds good to me, although it might be a courtesy to add some kind
of a field in the forensics.commands.command object which specifies
which version of volatility is needed for the plugin, just so that uses
end up with a "this plugin won't work with this version" message, rather
than a python stacktrace. 5;) That'd need some planning to figure out
though (since there may be times a version bumps without breaking
compatibility).
It is not a drop in replacement at all - Hopefully its
easier to use
than the old Object code.
Ok, I was more just asking about code conversion, since the function
siganture for the Object constructor seemed similar to the call for
NewObject, I was wondering if changing the code was going to be as easy
as just changing all Objects to NewObjects. It sounds like I'm going to
have to know a lot more about how it works than I do at the moment to
start converting them (which also means the pylint patch may break some
things in a couple of places as I was doing the import clean-ups).
The new object code implements the
__getitem__ protocol so you can just do something
like:
task_info['image_file_name'] = task.ImageFileName or
'UNKNOWN'
task_info['process_id'] = task.UniqueProcessId.v() or -1
task_info['active_threads'] = task.ActiveThreads or -1
task_info['inherited_from'] =
task.InheritedFromUniqueProcessId.v() or -1
task_info['handle_count'] = task.ObjectTable.HandleCount or -1
task_info['create_time'] = task.CreateTime
If the pointer is NULL it returns a NoneObject which can be used as
the LHS of or to continue assignment (giving defaults).
Sounds pretty ideal though. 5;)
read_string and friends should be deprecated since the
act of reading
from struct members is now fully implemented by the struct members
themselves - not by an external function.
There's still a few places where this is in use and is just reading a
'char', rather than a specifically designated type
(linux.scan/win32.scan2). I think that's what I was asking, will it be
possible to fold in default types (like 'char', 'int', etc) into the
NewObject model, so that we can effectively drop forensics.object, and
then later rename forensics.object2 to its rightful place. 5:)
BTW. The stuff defined in overlay.py can be thought of
basically as
"fixups" to the vtypes (which are autogenerated from .h files).
Ah, yeah, I was wondering where the autogenerator code is (or are you
just saying it could be autogenerated from .h files? Presumably that
should be somewhere in the repository? How does it decide which classes
to yoink from the .h, or does it grab all of them and you weed them out
manually?
Note the lambda in the args for the String class - it
gets evaluated
at construction time and has access to the new struct, so it can
easily return a String with length x.NameLength. Therefore we dont
need a special class override for _CM_KEY_NODE to call a read_string
function.
Yep, that's a nice little trick. 5:)
i dont like using the "from xxx import *"
form because its messy and
pollutes the namespaces. We should use something like "import x.y.z as
z" to make things explicit and also keep module reference short.
I'm completely in agreement with you there. Does anyone have any
problems with converting the code to a style like that?
The plugin architecture (which came from PyFlag
originally) is used to
make management of the code much easier for large project - not so
much to allow other people to maintain their own plugins seperate from
the codebase. I think long term compatibility will be a huge issue and
a maintenance nightmare - we really need to bring all modules to the
same tree. In PF we can add new plugin directories via the --plugins
directive (I think this is already in the volatiltiy version of the
registry code).
Well, it will become difficult, but other large projects do manage
plugins (although granted, developers often have more time to work on
them that our userbase does). This comes back to the compatibility flag
in the plugin. As long as it can safely be discarded if it doesn't work
properly, that should be fine. Bringing modules into the tree is great,
but only if there's someone there to maintain them, otherwise you're
just burdening the developers with access to the tree with more code to
maintain. That's why planning a really good interface for everything,
and then trying hard not to change it, is a worthwhile challenge...
Yeah I should probably start documenting it a bit more
.... Its
probably stopping people from adopting it right now :-)
Well, I'm having a go at it, but I think it's mostly the mix of the two
systems that's hindering adoption. Not knowing whether you can or can't
use read_obj, etc and then whether you can use forensics.addrspace. The
documentation will certainly help, particularly if it shows how to
properly use the latest system for everything, though... 5;)
Mike 5:)