[Vol-dev] Upcoming version of volatility

Hi All, As you may be aware I have been working on the new upcoming version of volatility. It has been a massive cleanup of the code base and a rewrite of much of the underlying architecture. The main thrust of this release is the development of a framework, which can easily be integrated into other tools as a library. This mean removing all globals, removing command line flags and removing any automated magical stuff that happens behind the scenes. Redesign of the object model so its more consistent with a library, and style cleanup of the codebase to improve its quality. The following is the summary of the main architectural changes: - Domain specific profiles. The profiles are no longer associated with the address spaces. You can have any number of profiles alive at the same time (e.g. kernel space and user space). Profiles are simply templates for instantiating CTypes on an address space. Therefore a profile is not specifically tied into a particular address space in any way. This also removes the obj_vm/native_vm confusion for dereferencing across address spaces, as address space switching is now done explicitly. - Removal of VolatilityMagic - there are basically two use cases for these - the first is just a profile specific constant, and the second is the automated running of code to retrieve a value (e.g. kdbg scan). Constants are now parts of the profile itself (for example in linux the system map is stored as profile constants), and automated code running is removed in favor of explicit plugin execution (which is made easier now). - The plugin architecture is updated - Previously it was very hard to share code between plugins - this is now very simple. For example, any plugin can call any other plugin. Plugins also export more intelligent functions than calculate() - so they can be used as modules themselves. Plugins are now also selectable according to the profile - for example we have a linux "pslist" and a windows "pslist", so just calling plugins.pslist() will get the right module depending on the selected profile. - The registry code is updated to remove automatic importation of files. This allows for much simpler and more efficient packaging as core modules are referenced directly. Loading of external plugins can now be done via a Load() plugin. This architecture is better for use in a framework. - The main user interface is based on ipython. This makes the user interface much easier to use - with command line completion helping users throughout. Users can also run arbitrary code on the results and we now can have light weight extension scripts - without needing to write a proper plugin. http://code.google.com/p/volatility/wiki/ScudettesBranch has some information about the new interface, but I will post a screencast soon that shows more. - Style cleanups. This refactor cleans up the style. In particular issues such as lines over 80 chars and legacy calls to base class methods (i.e. not using super) are cleaned up. - The new release includes memory acquisition tools as well as linux support. What should we do about it? The current version is a major rewrite of the current code base - it is not a patch to the current codebase and retrofitting it would be too difficult. We need to port the existing modules to the new code base and clean up this code. All developers need to consider the changes present in this version and how they apply to the next volatility release. If people have general architectural suggestions/disagreements etc we need to resolve these now. We then need to create a new volatility 3.0 branch and complete the porting of the modules to it. I have made a spreadsheet to document what modules are converted to the new framework and which are not done yet: https://docs.google.com/spreadsheet/ccc?key=0Athc84IflFGbdHpGcmNjWFkycFN4Q0… I am making slow progress on porting the code over to the new code base. I am happy with the way things are looking, and the new code is looking very nice. If people want to help, please let me know and I will provide editing rights and you can put your name next to each item to take ownership. I suggest all major new development to go to this branch (such as new modules etc). I am looking to complete a release candidate in the next month or two, as there is really not that much left to do. I will be running a workshop at DFRWS about volatility and will want to have the new release definitely out by then. Thanks, Michael.