Hey Pasquale,
So the "ValueError: too many values to unpack" error that you've been
getting in both volatility branches (although one branch masks the full
backtrace and the other includes it) is due to using /proc/kallsyms. If you
look at the line right before the error:
(str_addr, symbol_type, symbol) = line.strip().split()
There are supposed to be three space separated values in the symbol file.
Now if you cat your /proc/kallsyms:
# cat /proc/kallsyms
....
c02f6680 r __param_disable
c02f6694 r __param_pool_mode
c02f66a8 R __stop___param
c02f7000 D _etext
00000000 a tcp.c [lime]
bf000000 t $a [lime]
bf000160 t $d [lime]
bf000168 t $a [lime]
bf0001e8 t $d [lime]
bf0001ec t $a [lime]
In the lines attributed to lime, there are actually 4 values, which
raises the ValueError. Aside from this problem, and despite the name
"kallsyms", the /proc/kallsyms file doesn't actually contain
"all" symbols.
For example:
$ grep init_task ~/source/kernel/goldfish/System.map
c02e3000 r __ksymtab_init_task
c02e91f8 r __kstrtab_init_task
c02fae08 D init_task
c031afe8 B init_task_group
# grep init_task /proc/kallsyms
c02e3000 r __ksymtab_init_task
c02e91f8 r __kstrtab_init_task
So compared to the kernel's System.map, the /proc/kallsyms is actually
missing many symbols. I would suggest building your Volatility profile with
the kernel's System.map in this case. Only as a last resort use
/proc/kallsyms and even in that case, delete the lines related to lime so
it doesn't raise ValueError.
MHL
On Fri, Feb 22, 2013 at 11:12 AM, Pasquale Stirparo <pstirparo(a)gmail.com>wrote:
Ok, trying to get my head around this. I get the
same error with both
vol-2.3_alpha and the latest volatility version available from git
repository. There are different erros for different plugins, of course. I
started considering that, although if I run strings on the dump I find the
data I'm looking for, there could be an error in the dump especially after
the error from the linux_arp (below). So running hexdump -C on both my
test-dump and the sample-Evo4G one, mine it looks a bit weird even though
starts with the right LiME signature. I attached in the txt file the first
lines of hexdump, while here there are the two error messages.
Here the two from linux_arp and linux_pslist. Any clue?
hydra:volatility-read-only paco$ python vol.py --profile=LinuxHTCOneXx86
-f /Users/paco/Documents/htconex.lime linux_arp
Volatile Systems Volatility Framework 2.3_alpha
No suitable address space mapping found
Tried to open image as:
MachOAddressSpace: mac: need base
LimeAddressSpace: lime: need base
WindowsHiberFileSpace32: No base Address Space
WindowsCrashDumpSpace64: No base Address Space
HPAKAddressSpace: No base Address Space
VirtualBoxCoreDumpElf64: No base Address Space
VMWareSnapshotFile: No base Address Space
WindowsCrashDumpSpace32: No base Address Space
JKIA32PagedMemoryPae: No base Address Space
AMD64PagedMemory: No base Address Space
JKIA32PagedMemory: No base Address Space
IA32PagedMemoryPae: Module disabled
IA32PagedMemory: Module disabled
FileAddressSpace - EXCEPTION: too many values to unpack
ArmAddressSpace: No base Address Space
hydra:Volatility 2.3-devel paco$ python vol.py --profile=LinuxHTCOneXx86
-f /Users/paco/Documents/htconex.lime linux_pslist
Volatile Systems Volatility Framework 2.3_alpha
Traceback (most recent call last):
File "vol.py", line 186, in <module>
main()
File "vol.py", line 177, in main
command.execute()
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/plugins/linux/common.py", line 51, in execute
commands.Command.execute(self, *args, **kwargs)
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/commands.py", line 111, in execute
func(outfd, data)
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/plugins/linux/pslist.py", line 60, in render_text
("Start Time", "")])
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/commands.py", line 172, in table_header
profile = addrspace.BufferAddressSpace(self._config).profile
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/addrspace.py", line 169, in __init__
BaseAddressSpace.__init__(self, None, config, **kwargs)
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/addrspace.py", line 71, in __init__
self.profile = self._set_profile(config.PROFILE)
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/addrspace.py", line 94, in _set_profile
ret = profs[profile_name]()
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/plugins/overlays/linux/linux.py", line 145, in __init__
obj.Profile.__init__(self, *args, **kwargs)
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/obj.py", line 857, in __init__
self.reset()
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/plugins/overlays/linux/linux.py", line 156, in reset
self.load_sysmap()
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/plugins/overlays/linux/linux.py", line 199, in
load_sysmap
_memmodel, sysmapvar = parse_system_map(sysmapdata, "kernel")
File
"/Users/paco/Documents/Projects/MobiLeak/Paper2-memory/memdump/sample-Rodeo2012/Volatility
2.3-devel/volatility/plugins/overlays/linux/linux.py", line 86, in
parse_system_map
(str_addr, symbol_type, symbol) = line.strip().split()
ValueError: too many values to unpack
On Fri, Feb 22, 2013 at 4:20 PM, Pasquale Stirparo <pstirparo(a)gmail.com>wrote:
Thanks Michael,
I had run
$ python vol.py --info
but obviously with all the info output I somehow did not see the
different name of the htc profile... unbelievable but true :(
Anw thanks, it actually gives the output
Profiles
--------
...
LinuxHTCOneXx86 - A Profile for Linux HTCOneX x86
But now I'm getting other errors... each different for different
plugins I'm trying :(
I'll try to understand where is the problem and come back to the list.
Thanks again
On Fri, Feb 22, 2013 at 3:59 PM, Michael Hale Ligh <
michael.hale(a)gmail.com> wrote:
> Hi Pasquale,
>
> Can you run this command:
>
> # vol.py --info | grep Profile
>
> The profile may not actually be named HTCOneX exactly. For example I
> have a Ubuntu1204.zip but the profile name is a little different:
>
> $ python vol.py --info | grep Profile
> Volatile Systems Volatility Framework 2.3_alpha
> ....
> LinuxUbuntu1204x64 - A Profile for Linux Ubuntu1204 x64
> .....
>
> So I would need to use --profile=LinuxUbuntu1204x64
>
>
> On Wed, Feb 20, 2013 at 11:55 AM, Pasquale Stirparo <
> pstirparo(a)gmail.com> wrote:
>
>> Sorry guys... somehow I manage to push the keyboard shortcut and send
>> the email before finishing. I'm sending it again compleet now.
>>
>>
>> Hi Andrew,
>>
>> A summary hopefully useful also for others (especially mac users)
>> 1) Thanks for the link related to dwarfdump, it worked indeed. It
>> requires some dependancies, particularly for Mac OS X users libelf and
>> elf.h (you can find it attached). This are the steps I followed:
>> - install libelf (you can download it from here
>>
http://freecode.com/projects/libelf)
>> - copy the file elf.h (attached) to /usr/include
>> - compile dwarfdump (
http://reality.sgiweb.org/davea/dwarf.html)
>> - There is no make install, the make will compile the different
>> tools. The developer suggest to use the new version
>> .../dwarfdump2/dwarfdump. I'm also attaching the makefile sample updated to
>> this.
>>
>> 2) Thanks. As I wrote in my previous email, I had find out that if
>> exporting from the phone, on android you could find it under /proc/kallsyms
>> :)
>>
>> So, dwarfdump created a valid module.dwarf file and I did zip both it
>> and kallsyms in HTCOneX.zip, and placed it inside the overlays/linux folder
>> as follow (I've been trying using both volatility versions, the latest you
>> can get from git and the 2.3_alpha from DFWRS):
>> hydra:linux paco$ zip
>> /Users/paco/Documents/Projects/HTCOneX-vol-profile.zip module.dwarf
>> /Users/paco/Documents/Tools/HTC-OneX/endeavoru-2.6.39-6b459dc/kallsyms
>> adding: module.dwarf (deflated 90%)
>> adding:
>> Users/paco/Documents/Tools/HTC-OneX/endeavoru-2.6.39-6b459dc/kallsyms (deflated
>> 75%)
>>
>> hydra:linux paco$ cp
>> /Users/paco/Documents/Projects/HTCOneX-vol-profile.zip
>> /Users/paco/Documents/Projects/sample-Rodeo2012/Volatility\
>> 2.3-devel/volatility/plugins/overlays/linux/HTCOneX.zip
>>
>> hydra:linux paco$ cp
>> /Users/paco/Documents/Projects/HTCOneX-vol-profile.zip
>>
/Users/paco/Documents/Tools/volatility/volatility-read-only/volatility/plugins/overlays/linux/HTCOneX.zip
>>
>> But when I run it I get still the error:
>> # python vol.py --profile=HTCOneX -f
>> /Users/paco/Documents/Projects/memdump/20130213-2-htconex.lime linux_pslist
>> ...
>> ERROR : volatility.addrspace: Invalid profile HTCOneX selected
>>
>>
>> So chatting on the IRC channel with udgover, he suggested to have a
>> look at volatility/plugins/overlays/linux.py file. In this file there
>> is LinuxProfileFactory where it checks for "system.map" file. So
I've tried
>> to add the same routine for kallsyms. But it didn't work. I even tried to
>> rename kallsyms to System.map and then zip it with module.dwarf. Still the
>> same error.
>>
>> I'll keep trying, if you have any clue in the meantime of the reason
>> of the above error... here I am :)
>> Thanks guys
>>
>> P.
>>
>>
>>
>> On Wed, Feb 20, 2013 at 3:42 AM, Andrew Case <atcuno(a)gmail.com>wrote:
>>
>>> Hello,
>>>
>>> Sorry for taking so long to respond...
>>>
>>> There are a couple issues:
>>>
>>> 1) You need to use dwarfump from
>>>
http://reality.sgiweb.org/davea/dwarf.html , not the os x supplied
>>> dwarfdump for Linux support.
>>>
>>> 2) You can get System.map values from a few places. First, check if
>>> the source distributed for your kernel has it. Second, see if the
>>> phone exports /proc/kallsyms. If neither of those work then we will
>>> have to find another solution.
>>>
>>> Thanks,
>>> Andrew
>>>
>>> On Sat, Feb 16, 2013 at 7:46 AM, Pasquale Stirparo <
>>> pstirparo(a)gmail.com> wrote:
>>> > Thanks again Andrew for the assistance and for the Makefile. So
>>> here a brief
>>> > summary for creating the Android profile, following the linux
>>> example here
>>> > (
https://code.google.com/p/volatility/wiki/LinuxMemoryForensics)
>>> >
>>> > DISCLAIMER: I'm using a Mac OS X system.
>>> >
>>> > - I modified the Makefile sent by Andrew, placed under
>>> > volatility/tools/linux/Makefile, adding the options for
>>> cross-compiling as
>>> > from the LiME kernel module. Basically the modifications are the
>>> following:
>>> >
>>> > CCPATH :=
>>> >
>>>
/path/to/android-ndk/toolchains/arm-linux-androideabi-4.4.3/prebuilt/darwin-x86/bin
>>> >
>>> > dwarf: module.c
>>> > $(MAKE) ARCH=arm CROSS_COMPILE=$(CCPATH)/arm-linux-androideabi- -C
>>> $(KDIR)
>>> > CONFIG_DEBUG_INFO=y M=$(PWD) modules
>>> >
>>> >
>>> > - The last instruction of the makefile "dwarfdump -di module.ko
>
>>> > module.dwarf" was originally with -di option. But -d appears to be
>>> not a
>>> > valid option anymore. Checking the man page, the alternatives for
>>> debugging
>>> > options are the following
>>> > --debug-abbrev, --debug-aranges, --debug-frame[=OFFSET],
>>> > --debug-info[=OFFSET], --debug-inlined,
>>> --debug-line[=OFFSET],
>>> > --debug-macinfo[=OFFSET], --debug-pubnames[=PATTERN],
>>> > --debug-pubtypes[=PATTERN], --debug-str
>>> > debug-info is the default (which looking at the output of "head
>>> > module.dwarf" seems to be the correct one), so i tried just to
>>> remove the -d
>>> > and also to put --debug-info without any offset value, but this is
>>> what I
>>> > get as content of the module.dwarf file
>>> > hydra:linux paco$ cat module.dwarf
>>> >
>>> ----------------------------------------------------------------------
>>> > File: module.ko (arm)
>>> >
>>> ----------------------------------------------------------------------
>>> > .debug_info contents:
>>> > < EMPTY >
>>> >
>>> > - The last step to make the profile is to zip the module.dwarf
>>> file and the
>>> > System.map of the kernel. After searching around, I found that
>>> System.map
>>> > file for android is /proc/kallsyms (probably you all already knew
>>> it, but I
>>> > didn't know this :) ).
>>> >
>>> >
>>> > Attached the make file I used, if all the rest of the steps I did
>>> are right,
>>> > it's missing the right debug option for the dwarfdump command.
>>> I've tried
>>> > also the others but no one seems to give the right output. Any
>>> tips on this?
>>> > Thanks
>>> >
>>> > P.
>>> >
>>> >
>>> >
>>> > On Fri, Feb 15, 2013 at 5:41 PM, Andrew Case <atcuno(a)gmail.com>
>>> wrote:
>>> >>
>>> >> Hello,
>>> >>
>>> >> We are currently testing a stripped down Makefile to help people
>>> >> compiling for different kernels than the one for the system they
>>> are
>>> >> on (which includes Android). Can you please try the attached
>>> makefile?
>>> >> You will need to change the KDIR varabile to point to your kernel
>>> >> headers or source.
>>> >>
>>> >> On Fri, Feb 15, 2013 at 5:55 AM, Pasquale Stirparo <
>>> pstirparo(a)gmail.com>
>>> >> wrote:
>>> >> > Hi All,
>>> >> >
>>> >> > I'm trying to make a profile for android device.
>>> >> > I did a memory dump with LiME of an HTC One X (Android 4.0.3,
>>> HTC Sense
>>> >> > 4.0,
>>> >> > kernel 2.6.39.4-g6b459dc).
>>> >> >
>>> >> > Now, following the instruction here
>>> >> >
https://code.google.com/p/volatility/wiki/LinuxMemoryForensics,
I was
>>> >> > trying to understand how to modify the makefile under
>>> >> > volatility/tools/linux/ , in order to point to my kernel
>>> source. The
>>> >> > thing
>>> >> > is that in from my kernel source folder I couldn't find a
>>> proper value
>>> >> > for
>>> >> > KDIR and KVER (although they should be pretty straightforward
>>> according
>>> >> > to
>>> >> > their name) that would fit with the path for make command as
>>> from the
>>> >> > following source code:
>>> >> >
>>> >> > pmem: pmem.c
>>> >> > $(MAKE) -C $(KDIR)/lib/modules/$(KVER)/build M=$(PWD)
>>> modules
>>> >> >
>>> >> > dwarf: module.c
>>> >> > $(MAKE) -C $(KDIR)/lib/modules/$(KVER)/build
>>> CONFIG_DEBUG_INFO=y
>>> >> > M=$(PWD) modules
>>> >> > dwarfdump -di module.ko > module.dwarf
>>> >> > $(MAKE) -C $(KDIR)/lib/modules/$(KVER)/build M=$(PWD)
>>> clean
>>> >> >
>>> >> >
>>> >> > Did anyone ever created an android profile? Any hint?
>>> >> > I've seen in the mailing list archive a thread
"Profile (ZIP)
>>> for
>>> >> > Android
>>> >> > 4.0.3" from Mike (in Cc), any news about that?
>>> >> >
>>> >> > Thank you
>>> >> >
>>> >> > P.
>>> >> >
>>> >> > --
>>> >> > Pasquale Stirparo, MEng
>>> >> > GCFA, OPST, OWSE, ECCE
>>> >> >
>>> >> > European Commission - JRC Joint Research Centre
>>> >> > Institute for the Protection and Security of the Citizen
(IPSC)
>>> >> > Digital Citizen Security Unit
>>> >> > Via E. Fermi, 2749 - TP 361
>>> >> > 21027 Ispra (VA) - Italy
>>> >> >
>>> >> > PGP Key: 0x4C589FB2
>>> >> > Fingerprint: 776D F072 3F43 D5DE CB55 86D2 55FF 14A7 4C58 9FB2
>>> >> >
>>> >> > Disclaimer: The views expressed are purely those of the writer
>>> and may
>>> >> > not
>>> >> > in any circumstance be regarded as stating an official
position
>>> of the
>>> >> > European Commission.
>>> >> >
>>> >> > _______________________________________________
>>> >> > Vol-users mailing list
>>> >> > Vol-users(a)volatilityfoundation.org
>>> >> >
http://lists.volatilityfoundation.org/mailman/listinfo/vol-users
>>> >> >
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Pasquale Stirparo, MEng
>>> > GCFA, OPST, OWSE, ECCE
>>> >
>>> > European Commission - JRC Joint Research Centre
>>> > Institute for the Protection and Security of the Citizen (IPSC)
>>> > Digital Citizen Security Unit
>>> > Via E. Fermi, 2749 - TP 361
>>> > 21027 Ispra (VA) - Italy
>>> >
>>> > PGP Key: 0x4C589FB2
>>> > Fingerprint: 776D F072 3F43 D5DE CB55 86D2 55FF 14A7 4C58 9FB2
>>> >
>>> > Disclaimer: The views expressed are purely those of the writer and
>>> may not
>>> > in any circumstance be regarded as stating an official position of
>>> the
>>> > European Commission.
>>>
>>
>>
>>
>> --
>> Pasquale Stirparo, MEng
>> GCFA, OPST, OWSE, ECCE
>>
>> European Commission - JRC Joint Research Centre
>> Institute for the Protection and Security of the Citizen (IPSC)
>> Digital Citizen Security Unit
>> Via E. Fermi, 2749 - TP 361
>> 21027 Ispra (VA) - Italy
>>
>> PGP Key: 0x4C589FB2
>> Fingerprint: 776D F072 3F43 D5DE CB55 86D2 55FF 14A7 4C58 9FB2
>>
>> Disclaimer: The views expressed are purely those of the writer and
>> may not in any circumstance be regarded as stating an official position of
>> the European Commission.
>>
>> _______________________________________________
>> Vol-users mailing list
>> Vol-users(a)volatilityfoundation.org
>>
http://lists.volatilityfoundation.org/mailman/listinfo/vol-users
>>
>>
>
--
Pasquale Stirparo, MEng
GCFA, OPST, OWSE, ECCE
European Commission - JRC Joint Research Centre
Institute for the Protection and Security of the Citizen (IPSC)
Digital Citizen Security Unit
Via E. Fermi, 2749 - TP 361
21027 Ispra (VA) - Italy
PGP Key: 0x4C589FB2
Fingerprint: 776D F072 3F43 D5DE CB55 86D2 55FF 14A7 4C58 9FB2
Disclaimer: The views expressed are purely those of the writer and may
not in any circumstance be regarded as stating an official position of the
European Commission.
--
Pasquale Stirparo, MEng
GCFA, OPST, OWSE, ECCE
European Commission - JRC Joint Research Centre
Institute for the Protection and Security of the Citizen (IPSC)
Digital Citizen Security Unit
Via E. Fermi, 2749 - TP 361
21027 Ispra (VA) - Italy
PGP Key: 0x4C589FB2
Fingerprint: 776D F072 3F43 D5DE CB55 86D2 55FF 14A7 4C58 9FB2
Disclaimer: The views expressed are purely those of the writer and may
not in any circumstance be regarded as stating an official position of the
European Commission.