Re: [Vol-dev] [patch] Pylint improvements for new_object

Hi Mike, On Sun, Sep 20, 2009 at 11:30 AM, Mike Auty <mike.auty(a)gmail.com> wrote: Thanks for that - the patch didnt seem to make it for me (it ended up being a 14 byte bz2 file - maybe gmail stole it?). Not exactly - the new object scheme itself doesnt need these as these functions were merged into the new object code. However, since I still had the old object code lying around there might have been some references left over. I dont think they are used though. (linked lists are now handled automatically by overlaying a class _LIST_ENTRY(CType) definitions on top of the struct member - defined in memory_objects/Windows/xp_sp2.py). utils.py is in the svn tree (or it should be). Some of that work is already done - please have a look at the new plugins which are doing much the same things - I just left vmodules there as a guide as to whats left to do. For example get_pslist() is now powered by a rewritten pslist() in ./forensics/win32/tasks.py and is used in the module memory_plugins/pstree.py for example. I dont think backwards compatibility is a realistic goal we can have when we change the underlying framework. I am more happy with a linux style development model where everyone is free to have their own modules, but committing them to the main tree will ensure they get maintained as the code evolves. It is not a drop in replacement at all - Hopefully its easier to use than the old Object code. The new object code implements the __getitem__ protocol so you can just do something like: task_info['image_file_name'] = task.ImageFileName or 'UNKNOWN' task_info['process_id'] = task.UniqueProcessId.v() or -1 task_info['active_threads'] = task.ActiveThreads or -1 task_info['inherited_from'] = task.InheritedFromUniqueProcessId.v() or -1 task_info['handle_count'] = task.ObjectTable.HandleCount or -1 task_info['create_time'] = task.CreateTime If the pointer is NULL it returns a NoneObject which can be used as the LHS of or to continue assignment (giving defaults). read_string and friends should be deprecated since the act of reading from struct members is now fully implemented by the struct members themselves - not by an external function. As an example here is moyix's registry code: class _CM_KEY_NODE(Object): """Class representing a _CM_KEY_NODE Adds the following behavior: * Name is read as a string. """ def __new__(typ, *args, **kwargs): obj = object.__new__(typ) return obj # Custom Attributes def getName(self): return read_string(self.vm, types, ['_CM_KEY_NODE', 'Name'], self.offset, self.NameLength) Name = property(fget=getName) implementing getName. This is now replaced by the new object definition (from ./forensics/win32/overlay.py): '_CM_KEY_NODE' : [ None, { 'Signature' : [ None, ['String', dict(length=2)]], 'LastWriteTime' : [ None, ['WinTimeStamp', {}]], 'Name' : [ None, ['String', dict(length=lambda x: x.NameLength)]], }], So you dont need read_string() at all - the object itself knows how to do it. If x is a _CM_KEY_NODE then x.Name will return the string via instantiating the String() object on the right AS. BTW. The stuff defined in overlay.py can be thought of basically as "fixups" to the vtypes (which are autogenerated from .h files). Where we have None we dont override the vtypes definition, and if we have something we do. So the overlay can be read in conjunction with the vtypes definition: '_CM_KEY_NODE' : [ 0x50, { 'Signature' : [ 0x0, ['String', dict(length=2)]], 'Flags' : [ 0x2, ['unsigned short']], .... In this case I put the String definition into vtypes, but if it comes from an autogenerated header file parser it might have crap in there (like void* ) for example. In this case the overlay can fix it up. Note the lambda in the args for the String class - it gets evaluated at construction time and has access to the new struct, so it can easily return a String with length x.NameLength. Therefore we dont need a special class override for _CM_KEY_NODE to call a read_string function. i dont like using the "from xxx import *" form because its messy and pollutes the namespaces. We should use something like "import x.y.z as z" to make things explicit and also keep module reference short. The plugin architecture (which came from PyFlag originally) is used to make management of the code much easier for large project - not so much to allow other people to maintain their own plugins seperate from the codebase. I think long term compatibility will be a huge issue and a maintenance nightmare - we really need to bring all modules to the same tree. In PF we can add new plugin directories via the --plugins directive (I think this is already in the volatiltiy version of the registry code). Yeah I should probably start documenting it a bit more .... Its probably stopping people from adopting it right now :-) Thanks, Michael.