4:45 p.m.
Hey Pasquale,
So the "ValueError: too many values to unpack" error that you've been
getting in both volatility branches (although one branch masks the full
backtrace and the other includes it) is due to using /proc/kallsyms. If you
look at the line right before the error:
(str_addr, symbol_type, symbol) = line.strip().split()
There are supposed to be three space separated values in the symbol file.
Now if you cat your /proc/kallsyms:
# cat /proc/kallsyms
....
c02f6680 r __param_disable
c02f6694 r __param_pool_mode
c02f66a8 R __stop___param
c02f7000 D _etext
00000000 a tcp.c [lime]
bf000000 t $a [lime]
bf000160 t $d [lime]
bf000168 t $a [lime]
bf0001e8 t $d [lime]
bf0001ec t $a [lime]
In the lines attributed to lime, there are actually 4 values, which raises
the ValueError. Aside from this problem, and despite the name "kallsyms",
the /proc/kallsyms file doesn't actually contain "all" symbols. For
example:
$ grep init_task ~/source/kernel/goldfish/System.map
c02e3000 r __ksymtab_init_task
c02e91f8 r __kstrtab_init_task
c02fae08 D init_task
c031afe8 B init_task_group
# grep init_task /proc/kallsyms
c02e3000 r __ksymtab_init_task
c02e91f8 r __kstrtab_init_task
So compared to the kernel's System.map, the /proc/kallsyms is actually
missing many symbols. I would suggest building your Volatility profile with
the kernel's System.map in this case. Only as a last resort use
/proc/kallsyms and even in that case, delete the lines related to lime so
it doesn't raise ValueError.
MHL
On Fri, Feb 22, 2013 at 11:12 AM, Pasquale Stirparo <pstirparo(a)gmail.com>wrote:
Ok, trying to get my head around this. I get the same
error with both
vol-2.3_alpha and the latest volatility version available from git
repository. There are different erros for different plugins, of course. I
started considering that, although if I run strings on the dump I find the
data I'm looking for, there could be an error in the dump especially after
the error from the linux_arp (below). So running hexdump -C on both my
test-dump and the sample-Evo4G one, mine it looks a bit weird even though
starts with the right LiME signature. I attached in the txt file the first
lines of hexdump, while here there are the two error messages.
Here the two from linux_arp and linux_pslist. Any clue?
hydra:volatility-read-only paco$ python vol.py --profile=LinuxHTCOneXx86
-f /Users/paco/Documents/htconex.lime linux_arp
Volatile Systems Volatility Framework 2.3_alpha
No suitable address space mapping found
Tried to open image as:
MachOAddressSpace: mac: need base
LimeAddressSpace: lime: need base
WindowsHiberFileSpace32: No base Address Space
WindowsCrashDumpSpace64: No base Address Space
HPAKAddressSpace: No base Address Space
VirtualBoxCoreDumpElf64: No base Address Space
VMWareSnapshotFile: No base Address Space
WindowsCrashDumpSpace32: No base Address Space
JKIA32PagedMemoryPae: No base Address Space
AMD64PagedMemory: No base Address Space
JKIA32PagedMemory: No base Address Space
IA32PagedMemoryPae: Module disabled
IA32PagedMemory: Module disabled
FileAddressSpace - EXCEPTION: too many values to unpack
ArmAddressSpace: No base Address Space
hydra:Volatility 2.3-devel paco$ python vol.py --profile=LinuxHTCOneXx86
-f /Users/paco/Documents/htconex.lime linux_pslist
Volatile Systems Volatility Framework 2.3_alpha
Traceback (most recent call last):
File "vol.py", line 186, in <module>
main()
File "vol.py", line 177, in main
command.execute()
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/plugins/linux/common.py", line 51, in execute
commands.Command.execute(self, *args, **kwargs)
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/commands.py", line 111, in execute
func(outfd, data)
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/plugins/linux/pslist.py", line 60, in render_text
("Start Time", "")])
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/commands.py", line 172, in table_header
profile = addrspace.BufferAddressSpace(self._config).profile
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/addrspace.py", line 169, in __init__
BaseAddressSpace.__init__(self, None, config, **kwargs)
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/addrspace.py", line 71, in __init__
self.profile = self._set_profile(config.PROFILE)
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/addrspace.py", line 94, in _set_profile
ret = profs[profile_name]()
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/plugins/overlays/linux/linux.py", line 145, in __init__
obj.Profile.__init__(self, *args, **kwargs)
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/obj.py", line 857, in __init__
self.reset()
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/plugins/overlays/linux/linux.py", line 156, in reset
self.load_sysmap()
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/plugins/overlays/linux/linux.py", line 199, in
load_sysmap
_memmodel, sysmapvar = parse_system_map(sysmapdata, "kernel")
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/plugins/overlays/linux/linux.py", line 86, in
parse_system_map
(str_addr, symbol_type, symbol) = line.strip().split()
ValueError: too many values to unpack
On Fri, Feb 22, 2013 at 4:20 PM, Pasquale Stirparo <pstirparo(a)gmail.com>wrote:
Thanks Michael,
I had run
$ python vol.py --info
but obviously with all the info output I somehow did not see the
different name of the htc profile... unbelievable but true :(
Anw thanks, it actually gives the output
Profiles
--------
...
LinuxHTCOneXx86 - A Profile for Linux HTCOneX x86
But now I'm getting other errors... each different for different plugins
I'm trying :(
I'll try to understand where is the problem and come back to the list.
Thanks again
On Fri, Feb 22, 2013 at 3:59 PM, Michael Hale Ligh <
michael.hale(a)gmail.com> wrote:
--
Pasquale Stirparo, MEng
GCFA, OPST, OWSE, ECCE
European Commission - JRC Joint Research Centre
Institute for the Protection and Security of the Citizen (IPSC)
Digital Citizen Security Unit
Via E. Fermi, 2749 - TP 361
21027 Ispra (VA) - Italy
PGP Key: 0x4C589FB2
Fingerprint: 776D F072 3F43 D5DE CB55 86D2 55FF 14A7 4C58 9FB2
Disclaimer: The views expressed are purely those of the writer and may not
in any circumstance be regarded as stating an official position of the
European Commission.
Hi Pasquale,
Can you run this command:
# vol.py --info | grep Profile
The profile may not actually be named HTCOneX exactly. For example I
have a Ubuntu1204.zip but the profile name is a little different:
$ python vol.py --info | grep Profile
Volatile Systems Volatility Framework 2.3_alpha
....
LinuxUbuntu1204x64 - A Profile for Linux Ubuntu1204 x64
.....
So I would need to use --profile=LinuxUbuntu1204x64
On Wed, Feb 20, 2013 at 11:55 AM, Pasquale Stirparo <pstirparo(a)gmail.com
--
Pasquale Stirparo, MEng
GCFA, OPST, OWSE, ECCE
European Commission - JRC Joint Research Centre
Institute for the Protection and Security of the Citizen (IPSC)
Digital Citizen Security Unit
Via E. Fermi, 2749 - TP 361
21027 Ispra (VA) - Italy
PGP Key: 0x4C589FB2
Fingerprint: 776D F072 3F43 D5DE CB55 86D2 55FF 14A7 4C58 9FB2
Disclaimer: The views expressed are purely those of the writer and may
not in any circumstance be regarded as stating an official position of the
European Commission.
wrote:
Sorry guys... somehow I manage to push the
keyboard shortcut and send
the email before finishing. I'm sending it again compleet now.
Hi Andrew,
A summary hopefully useful also for others (especially mac users)
1) Thanks for the link related to dwarfdump, it worked indeed. It
requires some dependancies, particularly for Mac OS X users libelf and
elf.h (you can find it attached). This are the steps I followed:
- install libelf (you can download it from here
http://freecode.com/projects/libelf)
- copy the file elf.h (attached) to /usr/include
- compile dwarfdump (http://reality.sgiweb.org/davea/dwarf.html)
- There is no make install, the make will compile the different tools.
The developer suggest to use the new version .../dwarfdump2/dwarfdump. I'm
also attaching the makefile sample updated to this.
2) Thanks. As I wrote in my previous email, I had find out that if
exporting from the phone, on android you could find it under /proc/kallsyms
:)
So, dwarfdump created a valid module.dwarf file and I did zip both it
and kallsyms in HTCOneX.zip, and placed it inside the overlays/linux folder
as follow (I've been trying using both volatility versions, the latest you
can get from git and the 2.3_alpha from DFWRS):
hydra:linux paco$ zip
/Users/paco/Documents/Projects/HTCOneX-vol-profile.zip module.dwarf
/Users/paco/Documents/Tools/HTC-OneX/endeavoru-2.6.39-6b459dc/kallsyms
adding: module.dwarf (deflated 90%)
adding:
Users/paco/Documents/Tools/HTC-OneX/endeavoru-2.6.39-6b459dc/kallsyms (deflated
75%)
hydra:linux paco$ cp
/Users/paco/Documents/Projects/HTCOneX-vol-profile.zip
/Users/paco/Documents/Projects/sample-Rodeo2012/Volatility\
2.3-devel/volatility/plugins/overlays/linux/HTCOneX.zip
hydra:linux paco$ cp
/Users/paco/Documents/Projects/HTCOneX-vol-profile.zip
/Users/paco/Documents/Tools/volatility/volatility-read-only/volatility/plugins/overlays/linux/HTCOneX.zip
But when I run it I get still the error:
# python vol.py --profile=HTCOneX -f
/Users/paco/Documents/Projects/memdump/20130213-2-htconex.lime linux_pslist
...
ERROR : volatility.addrspace: Invalid profile HTCOneX selected
So chatting on the IRC channel with udgover, he suggested to have a
look at volatility/plugins/overlays/linux.py file. In this file there
is LinuxProfileFactory where it checks for "system.map" file. So I've
tried
to add the same routine for kallsyms. But it didn't work. I even tried to
rename kallsyms to System.map and then zip it with module.dwarf. Still the
same error.
I'll keep trying, if you have any clue in the meantime of the reason of
the above error... here I am :)
Thanks guys
P.
On Wed, Feb 20, 2013 at 3:42 AM, Andrew Case <atcuno(a)gmail.com> wrote:
> Hello,
>
> Sorry for taking so long to respond...
>
> There are a couple issues:
>
> 1) You need to use dwarfump from
> http://reality.sgiweb.org/davea/dwarf.html , not the os x supplied
> dwarfdump for Linux support.
>
> 2) You can get System.map values from a few places. First, check if
> the source distributed for your kernel has it. Second, see if the
> phone exports /proc/kallsyms. If neither of those work then we will
> have to find another solution.
>
> Thanks,
> Andrew
>
> On Sat, Feb 16, 2013 at 7:46 AM, Pasquale Stirparo <
> pstirparo(a)gmail.com> wrote:
> > Thanks again Andrew for the assistance and for the Makefile. So here
> a brief
> > summary for creating the Android profile, following the linux
> example here
> > (https://code.google.com/p/volatility/wiki/LinuxMemoryForensics)
> >
> > DISCLAIMER: I'm using a Mac OS X system.
> >
> > - I modified the Makefile sent by Andrew, placed under
> > volatility/tools/linux/Makefile, adding the options for
> cross-compiling as
> > from the LiME kernel module. Basically the modifications are the
> following:
> >
> > CCPATH :=
> >
> /path/to/android-ndk/toolchains/arm-linux-androideabi-4.4.3/prebuilt/darwin-x86/bin
> >
> > dwarf: module.c
> > $(MAKE) ARCH=arm CROSS_COMPILE=$(CCPATH)/arm-linux-androideabi- -C
> $(KDIR)
> > CONFIG_DEBUG_INFO=y M=$(PWD) modules
> >
> >
> > - The last instruction of the makefile "dwarfdump -di module.ko >
> > module.dwarf" was originally with -di option. But -d appears to be
> not a
> > valid option anymore. Checking the man page, the alternatives for
> debugging
> > options are the following
> > --debug-abbrev, --debug-aranges, --debug-frame[=OFFSET],
> > --debug-info[=OFFSET], --debug-inlined, --debug-line[=OFFSET],
> > --debug-macinfo[=OFFSET], --debug-pubnames[=PATTERN],
> > --debug-pubtypes[=PATTERN], --debug-str
> > debug-info is the default (which looking at the output of "head
> > module.dwarf" seems to be the correct one), so i tried just to
> remove the -d
> > and also to put --debug-info without any offset value, but this is
> what I
> > get as content of the module.dwarf file
> > hydra:linux paco$ cat module.dwarf
> >
> ----------------------------------------------------------------------
> > File: module.ko (arm)
> >
> ----------------------------------------------------------------------
> > .debug_info contents:
> > < EMPTY >
> >
> > - The last step to make the profile is to zip the module.dwarf file
> and the
> > System.map of the kernel. After searching around, I found that
> System.map
> > file for android is /proc/kallsyms (probably you all already knew
> it, but I
> > didn't know this :) ).
> >
> >
> > Attached the make file I used, if all the rest of the steps I did
> are right,
> > it's missing the right debug option for the dwarfdump command. I've
> tried
> > also the others but no one seems to give the right output. Any tips
> on this?
> > Thanks
> >
> > P.
> >
> >
> >
> > On Fri, Feb 15, 2013 at 5:41 PM, Andrew Case <atcuno(a)gmail.com>
> wrote:
> >>
> >> Hello,
> >>
> >> We are currently testing a stripped down Makefile to help people
> >> compiling for different kernels than the one for the system they are
> >> on (which includes Android). Can you please try the attached
> makefile?
> >> You will need to change the KDIR varabile to point to your kernel
> >> headers or source.
> >>
> >> On Fri, Feb 15, 2013 at 5:55 AM, Pasquale Stirparo <
> pstirparo(a)gmail.com>
> >> wrote:
> >> > Hi All,
> >> >
> >> > I'm trying to make a profile for android device.
> >> > I did a memory dump with LiME of an HTC One X (Android 4.0.3, HTC
> Sense
> >> > 4.0,
> >> > kernel 2.6.39.4-g6b459dc).
> >> >
> >> > Now, following the instruction here
> >> > https://code.google.com/p/volatility/wiki/LinuxMemoryForensics ,
> I was
> >> > trying to understand how to modify the makefile under
> >> > volatility/tools/linux/ , in order to point to my kernel source.
> The
> >> > thing
> >> > is that in from my kernel source folder I couldn't find a proper
> value
> >> > for
> >> > KDIR and KVER (although they should be pretty straightforward
> according
> >> > to
> >> > their name) that would fit with the path for make command as from
> the
> >> > following source code:
> >> >
> >> > pmem: pmem.c
> >> > $(MAKE) -C $(KDIR)/lib/modules/$(KVER)/build M=$(PWD)
> modules
> >> >
> >> > dwarf: module.c
> >> > $(MAKE) -C $(KDIR)/lib/modules/$(KVER)/build
> CONFIG_DEBUG_INFO=y
> >> > M=$(PWD) modules
> >> > dwarfdump -di module.ko > module.dwarf
> >> > $(MAKE) -C $(KDIR)/lib/modules/$(KVER)/build M=$(PWD)
> clean
> >> >
> >> >
> >> > Did anyone ever created an android profile? Any hint?
> >> > I've seen in the mailing list archive a thread "Profile (ZIP)
for
> >> > Android
> >> > 4.0.3" from Mike (in Cc), any news about that?
> >> >
> >> > Thank you
> >> >
> >> > P.
> >> >
> >> > --
> >> > Pasquale Stirparo, MEng
> >> > GCFA, OPST, OWSE, ECCE
> >> >
> >> > European Commission - JRC Joint Research Centre
> >> > Institute for the Protection and Security of the Citizen (IPSC)
> >> > Digital Citizen Security Unit
> >> > Via E. Fermi, 2749 - TP 361
> >> > 21027 Ispra (VA) - Italy
> >> >
> >> > PGP Key: 0x4C589FB2
> >> > Fingerprint: 776D F072 3F43 D5DE CB55 86D2 55FF 14A7 4C58 9FB2
> >> >
> >> > Disclaimer: The views expressed are purely those of the writer
> and may
> >> > not
> >> > in any circumstance be regarded as stating an official position
> of the
> >> > European Commission.
> >> >
> >> > _______________________________________________
> >> > Vol-users mailing list
> >> > Vol-users(a)volatilityfoundation.org
> >> > http://lists.volatilityfoundation.org/mailman/listinfo/vol-users
> >> >
> >
> >
> >
> >
> > --
> > Pasquale Stirparo, MEng
> > GCFA, OPST, OWSE, ECCE
> >
> > European Commission - JRC Joint Research Centre
> > Institute for the Protection and Security of the Citizen (IPSC)
> > Digital Citizen Security Unit
> > Via E. Fermi, 2749 - TP 361
> > 21027 Ispra (VA) - Italy
> >
> > PGP Key: 0x4C589FB2
> > Fingerprint: 776D F072 3F43 D5DE CB55 86D2 55FF 14A7 4C58 9FB2
> >
> > Disclaimer: The views expressed are purely those of the writer and
> may not
> > in any circumstance be regarded as stating an official position of
> the
> > European Commission.
>
--
Pasquale Stirparo, MEng
GCFA, OPST, OWSE, ECCE
European Commission - JRC Joint Research Centre
Institute for the Protection and Security of the Citizen (IPSC)
Digital Citizen Security Unit
Via E. Fermi, 2749 - TP 361
21027 Ispra (VA) - Italy
PGP Key: 0x4C589FB2
Fingerprint: 776D F072 3F43 D5DE CB55 86D2 55FF 14A7 4C58 9FB2
Disclaimer: The views expressed are purely those of the writer and may
not in any circumstance be regarded as stating an official position of the
European Commission.
_______________________________________________
Vol-users mailing list
Vol-users(a)volatilityfoundation.org
http://lists.volatilityfoundation.org/mailman/listinfo/vol-users