Hi Michael,

Thanks for your response.

I was not aware of the plugins by Carl Pulley. They include references
to the research, so that is definitely helpful.
Not so sure about pool/heap tags, I think I'll skip that for now.

I've already talked about something with Andrew W. in a private
conversation, namely the windows debug symbols. From the plugins you
linked, I gather that they are acquired from a webservice of sorts.
But the symbols.py plugin also seems to function without these debug
symbols. I found this comment:

"If use_symbols is False or no symbol information is available, then
module exports information is used to populate the symbols tables."

Since I am not very familiar with the windows world yet, do you know
what "module exports information" means?

The plugins do some more things I had hoped to do, such as finding ROP
chains. I could still implement it for linux, but it would not be all
that academically relevant anymore :)

I've had some time already after I sent this thread start, so I've
actually implemented finding the stack frames and connecting them to
function symbols (not debug symbols), using distorm3 to find the
relevant addresses. It works for my test program which calls both libc
and my own test library, so I'm pretty happy with that. Still needs a
lot of testing and polish though :)

Since it's looking more and more like my research won't be all that
new, I think I'll focus on a more broad context after this, drawing a
comparison to different compilers and options, and comparing the 4 big
operating systems (ie windows, linux, *bsd, os x).

Cheers,
Edwin

On 5 June 2013 19:42, Michael Hale Ligh <michael.hale@gmail.com> wrote:
> Hi Edwin,
>
> Sorry for the delay. Your research sounds very cool. I just have a few quick
> suggestions. First, you may already be aware of Carl Pulley's plugins [1] -
> in particular the exportstack.py and symbols.py which do similar tasks on
> Windows memory dumps. It may be a useful point of reference. Also, in the
> past I've mined heap headers for meta-data leading to clues on what the heap
> entries may contain. Its the same basic idea as kernel pool tag scanning
> etc, but in user mode memory. For example if you know the approximate size
> of the structure you're looking for, obviously you can eliminate any heap
> entries that are too small or large for storing them. In the Windows world,
> there is a poorly documented feature called heap tagging [2] where
> individual heap entries have associated tags to mark their contents, much
> like pool tags. You may look around for something similar in the Linux
> realm. Finally, if you have access to application symbols and the ability to
> use distorm3 stream disassembler, you might look for instructions in a
> binary that you know reference structures in process memory.
>
> Hope this helps some,
> MHL
>
> [1] https://github.com/carlpulley/volatility
> [2] http://www.alex-ionescu.com/?p=18
>
>
> On Fri, May 24, 2013 at 10:11 AM, Edwin Smulders <edwin.smulders@gmail.com>
> wrote:
>>
>> Hi developers and enthusiasts,
>>
>>
>> A while ago I introduced myself here, as I am doing a study into user
>> space memory forensics.
>>
>> My goal is to develop a general method to extract information from
>> user space, from any process. The past few months I have worked on
>> some proof of concepts, and some basic experimentation. This has all
>> been fairly easy, but I want to do something more complex. Today I am
>> mailing you to ask for some ideas and suggestions.
>>
>> To give you an idea of what I have done so far (and the level I'm
>> working on), here's a short list:
>>
>>  - Extract register contents from the process kernel stack
>>  - Determine the exact locations of stack frames
>>  - Make a list of all process threads and access their stacks
>>  - Walk a list of pointers and do heuristic search on small structs,
>> such as common networking structs
>> (NOTE: All using a linux dump)
>>
>> All of this worked fairly well, and it was fun, but not exactly
>> groundbreaking. I thought it would be best to ask the advice from
>> professionals before I continue.
>>
>> The direction I would like to take is doing something more foolproof
>> than a heuristic search, I would rather derive the location of
>> structures using information in memory.
>>
>> One idea I had was to identify the whole stack trace (i.e. including
>> function names) and use this to find struct locations. Since there is
>> always the issue of missing symbols, I would somehow have to
>> reintroduce these, and also figure out how dynamic symbols work. I am
>> especially interested in common libraries with this approach, since
>> something I can apply to libraries, I can apply to many processes.
>>
>> Another idea I had would be to use the source/headers of the
>> application, maybe similar to the way volatility uses the symbols for
>> the kernel.
>>
>>
>> Now I don't really have a question about this, I am just looking for
>> ideas. Please tell me if you think my ideas are good, or bad. If you
>> read this and wonder why I don't do X instead of Y, please ask me.
>> Feel free to mention any problems you see coming. I am really looking
>> for a discussion here. The only requirements I have at this moment are
>> that it stays roughly the same topic and it is academically relevant.
>> That last part also means that if you know about somebody doing
>> exactly this, do let me know, so I can do something else.
>>
>> If you know of any technical documentation or scientific research in
>> the area, I'm also interested in that. The most relevant paper (in the
>> whole topic of user space memory forensics) I have read so far is
>> "Digging for Data Structures", Cozzie et al (thanks emdel for showing
>> me that one).
>>
>> I permanently reside on IRC, Freenode/#volatility, if anyone cares to
>> chat some more about the topic.
>>
>>
>> Cheers,
>> Edwin
>> _______________________________________________
>> Vol-dev mailing list
>> Vol-dev@volatilesystems.com
>> http://lists.volatilityfoundation.org/mailman/listinfo/vol-dev
>
>