Adding kernel version / distro generic Linux support to Volatility (long email & code)
by Andrew Case
Hello,
With quite a bit of help & patience from ikelos, I have created a
skeleton for generic Linux support for Vol. In order to show the
complete process of how it currently works, I am going to demonstrate
the process from start to finish (a simple process listing of a memory
image). The main reason I am sending this email is because I hope
that the volatility developers can review what I have so far, comment,
and ultimately we can finalize a skeleton that can then be rapidly
added on to develop tens of useful plugins for Linux memory analysis.
I already have code for quite a bit of kernel analysis, so my whole
goal of this is to fit it inside the framework and ultimately get it
accepted into Volatility stable.
So the process of my system currently works like this (will be
illustrated with code and examples after):
1) Gather all of the structure offsets from a kernel through an LKM
2) Feed the LKM's output to a python script that generates a vtype /
profile automatically based on structure definitions kept in another
file
3) Make a profile python file for these types
4) Now we have a profile for the specific type and plugins can use it as normal
after this point we can run plugins, so now to illustrate all of this:
1) Gathering of offsets is facilitated in this LKM. (I added a simple
license & author header to each file due to ikelos's advice to do so)
http://gist.github.com/595600
This module works by gathering the offset of members from every
structure necessary to do analysis. It also adds the size of each
structure into the output file in order to have correct profile
information. Note that this code is kernel version aware and will be
able to get offsets from any 2.6.x version. Also, it only gathers a
small number of structures & members now b/c that is all that is
needed in order to list processes. Once the code is reviewed by devs,
other structures can be quickly added.
2) Feed this 1) to a python script
So at this step we need to generate the profile and we also need to
set the DTB. Even though it may seem odd, its not possible in a 2.6.x
generic way to get the cr3/dtb of the init process in order to
automatically acquite the dtb, even inside the kernel module. This is
because the DTB we want (swapper_pg_dir) is not exported and the mm
structure that references it (init_mm) stopped being exported in
2.6.26. Also, you can't get a reference to init_mm through init_task,
because after initialize init_task->mm is set to NULL. Because of all
this, we have to accept the DTB value on the commandline to this
script, but this is easily obtained by grepping for 'swapper_pg_dir'
in the corresponding System.map file for the kernel.
Here is the python script to generate the output, note that the
linux_kernel_types file will take work as well to support all
structures & members needed for full analysis, but that it will be
stable for all of 2.6.x once set:
http://gist.github.com/595589
http://gist.github.com/595598
Here is using it on the test kernel:
http://gist.github.com/595594
as you can see it has all the kernel structures and uses
volatility_magic to store the DTB. This means the users never have to
deal with DTB and it will still with the profile forever and will auto
load on vol initialization
now that we have this:
3) make a simple python profile
With the help of ikelos, I have made the profile system very easy to
add support for new kernel versions. To support this there is an
abstract Linux Profile that does everything but set the
abstract_types:
http://gist.github.com/595604
and the simple profile needed to support the debian test image:
http://gist.github.com/595606
4) time to write a plugin!
this is the simple plugin I came up with to list processes & it uses
profile just generated:
http://gist.github.com/595607
note that the system map parsing stuff will go into a utility file
once devs approve of the current way of doing things or we change
stuff around. its very quick to parse so its not an issue to parse it
on every load or eventually have the system map hash table pickled in
the cache
With that, we have distro / version generic Linux support. We also
have an easy method to generate profiles for any Linux System.
Please send any comments / suggestions, as once its approved I would
like to quickly add support for a large number of more plugins &
kernels
14 years, 1 month
- 2
- 3
Follow up to Linux support (read other email first) - a number of discussion points.....
by Andrew Case
I didn't want to put this in the last email as it wasn't exactly related.
I currently see two (non-technical) issue with full blown Linux support.
1) Since there are now 36 stable 2.6.x kernel versions and ten or so
distros, even partial coverage will require a huge amount of profiles.
While its easy to install a bunch of distros & kernels in a VM and get
profiles for them, the big problem will be how to allow users to
choose what profile is right without overwhelming them with options.
It is no where near as simple as Windows with relatively few versions
compared to Linux.
2) Going off 1), there is also the problem of how to name all the
profiles and the eventual --profile option. Since distros have
multiple kernels packaged for the same kernel version its not as
simple as what I have now (LinuxDebian2626) because debian has
multiple 2626 packages, that will all have different offsets, dtb, and
such. So we need to come up with a sane method of allowing all the
profiles to be choosen.
What I was thinking for this is a complete reorder of the --help
system, which honestly is confusing now. Since vol will now have
windows & linux support, and (soon ;)) will have support for some
specific applications memory analysis, I think the help menu should be
something like this (close to what svn does)
python volatility -h Windows
python volality -h Linux
python volatility -h some_app
and then it will show you only the plugins for whichever os/app you
are interested in. This can be made simple inside the current
framework, by tagging each plugin with what they support inside of the
string message. so for instance you can have:
class some_linux_plugin(...):
''' Linux - A module for linux process listing '''
class some_windows_plugin(...):
''' Windows - A module for windows process listing '''
and this way only the stuff after - gets displayed and the parser
knows what plugins work on which apps.
for plugins that work regardless of OS there could be a "-h any"
option or similar...
3) To help with profile explosion, there also should be some way to
list profiles for a specific app/os and for linux, which distro:
as of right now, the only way I know of to get a list of profiles is
with --info, which also lists a bunch of other stuff. If there was a
way to do something like:
python volalitility.py --list-profiles Windows
which wouuld return all windows profiles
or
--list-profiles Windows XP
which would return profiles for windows xp
similarly for linux you could have:
--list-profiles Linux
--list-profiles Linux Debian
which would greatly help if we are going to properly support Linux &
all the different distros and versions
Even without Linux support, this issue will crop up as volality moves
to support XP (with 3 service packs), vista (with 2 currently), 7,
2008, 2003, etc as relying on --info will just result in huge blocks
of text
14 years, 1 month
- 1
- 0