(started 4/5/2008, added 5/5/2008)
This document describes how it could feasible to implement a pseudo
PaX implementation, completely in userland. The
described idea is far more of a play thing, than anything completely serious,
for reasons later described. It's more of just random thoughts and
experiments.
I don't recall how I got started along this track, except that it was
something I've been meaning to look at for a while.
Introduction
Firstly, we should review how PaX's
segmexec operates.
While Linux effectively does not use segmentation by creating 0 based and
4 GB limited segments for both code and data accesses (therefore logical
addresses are the same as linear addresses), it is possible to set up
segments that allow to implement non-executable pages.
The basic idea is that we divide the 3 GB userland linear address space
into two equal halves and use one to store mappings meant for data access
(that is, we define a data segment descriptor to cover the 0-1.5 GB linear
address range) and the other for storing mappings for execution (that is,
we define a code segment descriptor to cover the 1.5-3 GB linear address
range). Since an executable mapping can be used for data accesses as well,
we will have to ensure that such mappings are visible in both segments
and mirror each other. This setup will then separate data accesses from
instruction fetches in the sense that they will hit different linear
addresses and therefore allow for control/intervention based on the access
type. In particular, if a data-only (and therefore non-executable) mapping
is present only in the 0-1.5 GB linear address range, then instruction
fetches to the same logical addresses will end up in the 1.5-3 GB linear
address range and will raise a page fault hence allow detecting such
execution attempts.
PaX's segmexec works by modifying the
Global Descriptor Table
which separates code and data requests to different virtual addresses.
Userspace, as far as I know, can't modify the Global Descriptor Table, but it
can influence it's own
Local Descriptor Table
via the modify_ldt() system call. The modify_ldt() syscall can create code
and data descriptors easily enough, and we can use call far and return far
(amongst other techniques) to change into that selector.
Proof of concept
As a sample, let's try and execute a int3 instruction. We'll create a new LDT
entry, with a base address of 4096, which means all CS addresses after that's
set, has to be subtracted by 4096. And on with the show:
#include <stdlib.h>
#include <unistd.h>
#include <strings.h>
#include <stdio.h>
#include <errno.h>
#include <sys/types.h>
#include <fcntl.h>
#include <asm/ldt.h>
asm(".globl debug;\
.type debug, @function;\
debug:;\
int3;\
.size exit, .-exit;\
"
);
extern void debug();
/*
<asm/ldt.h>
struct modify_ldt_ldt_s {
unsigned int entry_number;
unsigned long base_addr;
unsigned int limit;
unsigned int seg_32bit:1;
unsigned int contents:2;
unsigned int read_exec_only:1;
unsigned int limit_in_pages:1;
unsigned int seg_not_present:1;
unsigned int useable:1;
};
#define MODIFY_LDT_CONTENTS_DATA 0
#define MODIFY_LDT_CONTENTS_STACK 1
#define MODIFY_LDT_CONTENTS_CODE 2
*/
int do_ldt(int num, unsigned long base, int type)
{
struct modify_ldt_ldt_s ldt_entry = {
num, // entry_number
(unsigned long int) (base), // base_address
0xfffff, // limit, 4G or so :p
1, // seg_32bit
type, // contents
1, // read_exec_only
1, // limit_in_pages
0, // seg_not_present
1 // usable
};
return modify_ldt(1, &ldt_entry, sizeof(struct modify_ldt_ldt_s)) == 0;
}
int main(int argc, char **argv)
{
short int seg;
if(do_ldt(0, 0x1000, MODIFY_LDT_CONTENTS_CODE) == 0) {
printf("Failed to modify ldt\n");
exit(EXIT_FAILURE);
}
seg = 7; // (0 * 8) + 7
//printf("new segment: %d|%02x\n", seg, seg);
__asm__ volatile("pushw %0;\
pushl %1;\
lret"
:
: "r" (seg), "r" ((unsigned int)(debug) - 0x1000)
);
}
Running the above code under a debugger:
Copyright (C) 2007 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...
(no debugging symbols found)
Using host libthread_db library "/lib/libthread_db.so.1".
(gdb) r
Starting program: /root/drifter/ldt/ldt_wot
(no debugging symbols found)
(no debugging symbols found)
Program received signal SIGTRAP, Trace/breakpoint trap.
0x08047559 in ?? ()
(gdb) x/4i $eip -1
0x8047558: Cannot access memory at address 0x8047558
(gdb) x/4i $eip - 1 + 4096
0x8048558 <debug>: int3
0x8048559 <do_ldt>: push %ebp
0x804855a <do_ldt+1>: mov %esp,%ebp
0x804855c <do_ldt+3>: sub $0x48,%esp
(gdb) i r cs
cs 0x7 7
As we can see in the debugger output, it's possible to set a custom CS
descriptor, and execute code. Due to the now non-flat memory address space,
it also messes with debugging a little bit.
Implementing the Pseduo-PaX
Quickly reviewing what we need to do:
-
Choose where to split our memory (let's stick with 0x60000000)
-
Duplicate code to the code section
-
Handle segmentation violations
-
Unmap old stack because it's at an inconvenient location.
Doing the above completely correctly would be difficult from userland, but
possible if a bit of effort was to be expended.
For the purposes of this article, we'll write it using dietlibc, and make it
not that feasible to use.
#include <stdlib.h>
#include <unistd.h>
#include <strings.h>
#include <stdio.h>
#include <errno.h>
#include <sys/types.h>
#include <fcntl.h>
#include <asm/ldt.h>
#include <sys/mman.h>
#include <asm/unistd.h>
/*
<asm/ldt.h>
struct modify_ldt_ldt_s {
unsigned int entry_number;
unsigned long base_addr;
unsigned int limit;
unsigned int seg_32bit:1;
unsigned int contents:2;
unsigned int read_exec_only:1;
unsigned int limit_in_pages:1;
unsigned int seg_not_present:1;
unsigned int useable:1;
};
#define MODIFY_LDT_CONTENTS_DATA 0
#define MODIFY_LDT_CONTENTS_STACK 1
#define MODIFY_LDT_CONTENTS_CODE 2
*/
_syscall3(int,modify_ldt,int,op,void*,what,int,len);
/*int modify_ldt(int op, void *what, int len)
{
return syscall(__NR_modify_ldt, op, what, len);
}*/
int do_ldt(int num, unsigned long base, int type)
{
struct modify_ldt_ldt_s ldt_entry = {
num, // entry_number
(unsigned long int) (base), // base_address
0x5ffff, // limit, 1.5G or so :p
1, // seg_32bit
type, // contents
1, // read_exec_only
1, // limit_in_pages
0, // seg_not_present
1 // usable
};
return modify_ldt(1, &ldt_entry, sizeof(struct modify_ldt_ldt_s)) == 0;
}
int do_exit()
{
exit(EXIT_SUCCESS);
}
void vulnerable()
{
int j[1];
int i;
char code[] = "\xcc\xcc\xcc\xcc";
#ifdef HEAP
int addr = strdup(code);
#else
int addr = &code;
#endif
//char *where;
//__asm__("int3;");
for(i = 0; i < 10; i++) j[i] = (unsigned int)(addr);
}
unsigned char *old_stack;
int old_stack_len;
void duplicate_code_mappings()
{
// taken from drifter level 11 code, but modified.
FILE *f;
int hi, low;
char flags[5];
int wot;
int major, minor;
int size;
char remainder[1024];
int ret;
unsigned char *new;
f = fopen("/proc/self/maps", "r");
while(8 == (ret = fscanf(f, "%08x-%08x %[^ \n] %08x %02x:%02x %08x%[^\n]", &low, &hi, flags, &wot, &major, &minor, &size, remainder))) {
if((low & 0x60000000) == 0x60000000) continue;
size = hi - low;
/*
* size = hi - low;
* printf("--> %08x-%d\n", low, size);
printf("--> %08x-%08x %s %d %d:%d %d %s\n", low, hi, flags, wot, major, minor, size, remainder);
*/
// r-xp
if(flags[1] == '-' && flags[2] == 'x') {
printf("--> Duplicating 0x%08x, %d bytes long\n", low, size);
new = mmap(low+0x60000000, size, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_FIXED|MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
if(new == MAP_FAILED) {
printf("Unable to map code duplicate: %s\n", strerror(errno));
exit(EXIT_FAILURE);
}
memcpy(new, low, size);
mprotect(new, size, PROT_READ|PROT_EXEC);
}
old_stack = low;
old_stack_len = size;
}
fclose(f);
//exit(EXIT_FAILURE);
}
#define STKSIZ (4096 * 32)
// more code from drifter level11.. so I'm lazy :P
unsigned char *allocate_stack()
{
int found;
int address;
short int shift;
unsigned char *stack_ptr;
int urand_fd;
urand_fd = open("/dev/urandom", O_RDONLY);
found = 0;
while(!found) {
if(read(urand_fd, &address, 4) != 4) {
printf("Read failure on /dev/urandom: %s\n", strerror(errno));
exit(EXIT_FAILURE);
}
if(read(urand_fd, &shift, 2) != 2) {
printf("Read failure on /dev/urandom: %s\n", strerror(errno));
exit(EXIT_FAILURE);
}
shift &= 4088; // (page_size - 1) - last 4 bits, to align stack
#if 1
address &= 0x5f7fffff; // remove everything except for last 8M of address space
//address |= TOOBIG;
#endif
address &= ~4095; // clear page addr
stack_ptr = mmap(address, STKSIZ, PROT_READ|PROT_WRITE, MAP_FIXED|MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
if(stack_ptr != MAP_FAILED) found = 1;
}
close(urand_fd);
stack_ptr = (unsigned int)(stack_ptr) + (STKSIZ) - shift;
memset(stack_ptr, 0xcc, shift);
stack_ptr--;
return stack_ptr;
}
void do_vulnerable()
{
munmap(old_stack, old_stack_len);
printf("Hello from do_vulnerable\n");
vulnerable();
do_exit();
}
int main(int argc, char **argv)
{
short int seg;
unsigned char *stack;
if(do_ldt(0, 0x60000000, MODIFY_LDT_CONTENTS_CODE) == 0) {
printf("Failed to modify ldt\n");
exit(EXIT_FAILURE);
}
seg = 7;
duplicate_code_mappings();
stack = allocate_stack();
printf("--> Will unmap old stack starting @ 0x%08x\n", old_stack);
system("cat /proc/$PPID/maps");
printf("Returning to do_vulnerable\n");
__asm__ volatile("movl %0, %%esp;\
movl %%esp, %%ebp;\
pushl %1;\
pushw %2;\
pushl %3;\
lret"
:
: "m"(stack), "m" (do_exit), "r" (seg), "r" ((unsigned int)(do_vulnerable))
);
}
The above was compiled with:
diet gcc -fno-pie -fno-stack-protector ldt_test.c -o ldt_test
The above code creates a new stack mapping (because the original is mapped
somewhere around 0xbfff0000) and allocates underneath the cutoff point, in
addition, it duplicates code layout via scanning /proc/self/maps for
mappings marked executable, and NOT writable.
The vulnerable() function stimulates a stack overflow, pointing to the int3
instruction. Optionally, it can be compiled with -DHEAP, and it will stimulate
a heap return address, rather than a stack return address.
Watching it catch heap execute attempts:
GNU gdb 6.7.1
Copyright (C) 2007 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...
(no debugging symbols found)
Using host libthread_db library "/lib/libthread_db.so.1".
(gdb) r
Starting program: /root/drifter/ldt/ldt_wot
--> Duplicating 0x08048000, 24576 bytes long
--> Will unmap old stack starting @ 0xbfffc000
00110000-00111000 rw-p 00000000 00:00 0
08048000-0804e000 r-xp 00000000 08:01 50639 /root/drifter/ldt/ldt_wot
0804e000-08050000 rw-p 00005000 08:01 50639 /root/drifter/ldt/ldt_wot
08050000-08051000 rwxp 00000000 00:00 0
0d2a4000-0d2c4000 rw-p 00000000 00:00 0
68048000-6804e000 r-xp 00000000 00:00 0
bfffc000-c0000000 rwxp ffffd000 00:00 0
Returning to do_vulnerable
Hello from do_vulnerable
Program received signal SIGSEGV, Segmentation fault.
0x00111008 in ?? ()
(gdb) x/4i $eip
0x111008: int3
0x111009: int3
0x11100a: int3
0x11100b: int3
And stack execution attempts:
GNU gdb 6.7.1
Copyright (C) 2007 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...
(no debugging symbols found)
Using host libthread_db library "/lib/libthread_db.so.1".
(gdb) r
Starting program: /root/drifter/ldt/ldt_wot
--> Duplicating 0x08048000, 24576 bytes long
--> Will unmap old stack starting @ 0xbfff4000
00110000-00111000 rw-p 00000000 00:00 0
063c3000-063e3000 rw-p 00000000 00:00 0
08048000-0804e000 r-xp 00000000 08:01 50635 /root/drifter/ldt/ldt_wot
0804e000-08050000 rw-p 00005000 08:01 50635 /root/drifter/ldt/ldt_wot
08050000-08051000 rwxp 00000000 00:00 0
68048000-6804e000 r-xp 00000000 00:00 0
bfff4000-c0000000 rwxp ffff5000 00:00 0
Returning to do_vulnerable
Hello from do_vulnerable
Program received signal SIGSEGV, Segmentation fault.
0x063e25b1 in ?? ()
(gdb) x/4i $eip
0x63e25b1: int3
0x63e25b2: int3
0x63e25b3: int3
0x63e25b4: int3
Implementing it properly
This idea would be easily implemented in the ELF loader (ld.so, not kernel) if
it laid out the memory correctly, and probably remapped the stack to a lower
address. It would also have to hook stuff like mmap() and duplicate it if
possible/applicable.
Anonymous memory could possibly be handled via making it disk backed, and
mapping twice from that.
As opposed to memcpy, it should correctly parse /proc/<pid>/maps, and mmap()
from shared libraries correctly.
Dynamically generated code could be handled by marking segments non writable,
when attempting to execute them. On attempts to write there again, it would be
unmapped from the executable region, and marked writable again. Self modifying
code would be handled via this mechanism, albeit slowly.
Weaknesses
-
Duplicating code section can't be completely the same, as
we can't duplicate anonymous mappings, etc, if they where
to become executable. A work around exists by making it
disk backed, however.
-
You could bypass it by returning to a retf instruction,
which would use the CS descriptor from the GDT, which is
probably marked 0-4G executable :p
-
Probably a bunch of other weaknesses.
-
Doing it from userland is pretty lame :P
Other uses for modify_ldt() syscall
-
Userland Process Scheduling (Another article for another day, maybe an
outline soon.)
-
Since debuggers on 32 bit platforms probably assume a flat memory layout,
it could be feasible to use different CS/DS/SS/whatever selectors as a
minor obfuscation mechanism.
(added 3/1/2008)
On the 3rd January, manio [at] skyboo [dot] net e-mailed me asking for some
hints / tips / advice about how the passwords are stored in the MikroTik
Router OS image. (To his credit, he said he realised it was XOR based pretty
much after he hit sent the mail). The user/password information is stored in
/nova/store/user.dat. His homepage is
http://manio.skyboo.net/mikrotik/.
According to him, the following passwords had the following encrypted text:
zero length pw 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0 78 BF DE 06 49 5A 0E 2D 09 D5 FB 27 B1 44 EC 93 01
aaa 29 D3 BF 06 49 5A 0E 2D 09 D5 FB 27 B1 44 EC 93 01
ala 29 DE BF 06 49 5A 0E 2D 09 D5 FB 27 B1 44 EC 93 01
0000 48 8F EE 36 49 5A 0E 2D 09 D5 FB 27 B1 44 EC 93 01
Initially, we can note that :
This made me think it was something trivial such as an XOR based scheme.
If it is, we can work out what the first XOR byte is by:
>>> hex(0x78 ^ ord('0'))
'0x48'
This works due to the properties of XOR.
Continuing on with our analysis / assumption that it is XOR on the
second char, we take the suspected xor byte of 0xbf, and XOR them against
the decimal value of a and l
>>> hex(0xbf ^ ord('a'))
'0xde'
>>> hex(0xbf ^ ord('l'))
'0xd3'
As we can see, the returned bytes are the same as the second bytes from the
"hash" from aaa and ala respectively.
Since we now know the "encryption" key, we can write a decoder trivially. (As
a side note, I like Python's doctest module :) )
$ python mikrotik_password.py 29 de bf 06 49 5a 0e 2d 09 d5 fb 27 b1 44 ec 93 01
aaa
The password decoder can be found here for those
who care.
I do not know if the encryption key changes on different releases of RouterOS,
or if it is dependant upon license key or anything like that - this was coded
with the information manio (lowercased upon his request) provided to me. manio
said that he would investigate this when he gets a chance.
(started 28/8/2007, added 11/10/2007)
The MikroTik Wireless Router is a
Linux embedded wireless router, focusing on various functionality such as
bandwidth management, Firewalling, VPN server/client, and various other
things. As with all embedded linux based software, it is interesting to pull
it apart :)
It has been around for a while now… a couple of years ago when I analysed
the software / pulling it apart, it had drivers/firmware to turn standard Orinoco
wireless cards into an Access Point (which as far as I know isn't possible
otherwise, at least not when I was looking at it.)
For the purposes of this article, I am looking at mikrotik-2.9.46.iso
(MD5sum: 65aa908dd748ccf72ad9f588613dfe31, SHA1sum:
5e5ed13498db8d9745a701f75e58da3ef6701e58). For the most part, I have used
QEMU to emulate the hardware/software
environment to install it on. This has several advantages, such as being able
to edit the "disk" it's using easily, amongst other things.
Performing active analysis of MikroTik router components
To perform more active analysis of the MikroTik components, we could copy the
applicable binaries and associated libraries to another linux platform. This
would allow us to strace the binary, debug it (which is incredibly useful
for exploit development), and monitor the activities it performs in general.
Furthermore, we can copy the kernel and applicable modules to perform further
analysis on them, and to allow the environment to be replicated a lot better.
The analysis environment / setup
For this article, I have done a basic network install of Debian 4rc1. After
performing the installation and installing a bunch of generic tools
(strace/gdb/gcc/ltrace/openssh-server/nasm/etc), I then extracted the Mikrotik kernel and
modules, and put the applicable files into their place.
[box] # wget http://felinemenace.org/~andrewg/MikroTik_Router_Security_Analysis_Part1/MikroTik-2.9.46-kernel-initrd.tgz
--08:58:00-- http://felinemenace.org/~andrewg/MikroTik_Router_Security_Analysis_Part1/MikroTik-2.9.46-kernel-initrd.tgz
=> `MikroTik-2.9.46-kernel-initrd.tgz'
Resolving felinemenace.org... 69.55.233.10
Connecting to felinemenace.org|69.55.233.10|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1,832,960 (1.7M) [application/x-gzip]
100%[====================================>] 1,832,960 327.68K/s ETA 00:00
08:58:06 (330.42 KB/s) - `MikroTik-2.9.46-kernel-initrd.tgz' saved [1832960/1832960]
[box] # tar xzf MikroTik-2.9.46-kernel-initrd.tgz
[box] # mv lib/modules/2.4.31/ /lib/modules/2.4.31
[box] # mv boot/vmlinuz /boot/vmlinuz-2.4.31
[box] # cd /boot/grub/
[box] # cat >>menu.lst <<_EOF_
> title MikroTik 2.4.31 / 2.9.46
> root (hd0,0)
> kernel /boot/vmlinuz-2.4.31
>
> _EOF_
[box] # sync ; reboot
While the QEMU image was rebooting, I wondered if it would work, due to
differences in 2.6 and 2.4 kernels.. Firstly though, it appeared I may have to
experiment with initrd images to load the applicable drivers for it and
generally mess around as when booting it displayed:
Booting 'MikroTik 2.4.31 / 2.9.46'
root (hd0,0)
Filesystem type is ext2fs, partition type is 0x83
kernel /boot/vmlinuz-2.4.31
[Linux-bzImage, setup=0x1400, size=0xa044d]
Uncompressing Linux... Ok, booting the kernel.
Kernel panic: VFS: Unable to mount root fs on 09:00
Booting back into the standard Debian kernel:
[box] # cd /boot/
[box] # cp /root/boot/initrd.rgz mikro-initrd.rgz
[box] # cd grub/
[box] # echo initrd /boot/mikro-initrd.rgz >>menu.lst
[box] # sync ; reboot
Unfortunately, this gives a similar error message before. We know previously
from mounting the mikrotik root filesystem it was ext3, but we can (attempt)
to verify what filesystems they support. Looking over the original find
output for the kernel modules, we don't see any filesystem modules for the
default ext2 filesystem.
We can verify the filesystems supported by analysing the vmlinuz file. To
summarise, basically, this file contains some bootup to get the machine into a
decent state, a gzip decompression routine, and a heap of compressed data. The
information we're interested in is in the compressed data, so we have to
decompress it. As it's not a standard gzip file, we can't just run gunzip on
it and be done with it, we need extract the compressed data.
Fortunately, this can be done rather easily because the the gzip header /
magic bytes, which allows us to find suitable offsets to attempt
decompression. The gzip magic bytes can be found by doing:
[box] # cd /tmp
[box] # cp /etc/passwd .
[box] # gzip passwd
[box] # xxd passwd.gz | head
0000000: 1f8b 0808 2061 d346 0003 7061 7373 7764 .... a.F..passwd
0000010: 0065 93c1 6ec2 300c 86ef 3c05 c74d 0285 .e..n.0...<..M..
0000020: 5260 90e3 3469 9771 d99e c06d 4289 d626 R`..4i.q...mB..&
0000030: 55d2 5278 fbd9 71a0 4593 adc8 7ffc 2536 U.Rx..q.E.....%6
0000040: 0ef5 ce75 f22a 5768 9e42 c16b 61ac 2820 ...u.*Wh.B.ka.(
0000050: 9c67 0a74 e32c 1219 5a12 a20f 5e04 4498 .g.t.,..Z...^.D.
0000060: 438a e2ab 5ca3 dd77 1fa9 700b 98ca d128 C...\..w..p....(
0000070: 124a 5f26 295b 626e 2377 db6d be91 514e .J_&)[bn#w.m..QN
0000080: cea2 9c55 d068 3abf 95bb 9564 11ab a730 ...U.h:....d...0
0000090: 5dd4 0095 dfc9 6c2d 2914 17f0 a284 f2ac ].....l-).......
For the purpose on hand, we'll use 0x1f8b as our marker.
[box] # xxd /boot/vmlinuz-2.4.31 | egrep "\b1f8b|1f\b \b8b" | head -n 5
00049a0: a9d0 0900 1f8b 0800 b533 bc46 0203 ec5d .........3.F...]
0004f20: 3613 e31f 8b6d a730 ef6f 1078 d415 4401 6....m.0.o.x..D.
001c6c0: 0a1f 8bab 69a2 a7e5 533b 4d60 764f 93bc ....i...S;M`vO..
00381b0: c3ba d727 7964 631f 8baf 810c 2704 206f ...'ydc.....'. o
003fdf0: 972f d2d6 d50e 5d37 180b 771f 8bc5 b43d ./....]7..w....=
To extract the the compressed data from the vmlinuz-2.4.31 file, dd does the
trick easily. We'll start from our first match and work our way down.
[box] # dd if=/boot/vmlinuz-2.4.31 of=vmlinuz.gz bs=1 skip=$((0x49a4))
662093+0 records in
662093+0 records out
662093 bytes (662 kB) copied, 16.0954 seconds, 41.1 kB/s
[box] # file vmlinuz.gz
vmlinuz.gz: gzip compressed data, from Unix, last modified: Fri Aug 10
19:45:25 2007, max compression
[box] # gunzip vmlinuz.gz
[box] # strings -a vmlinuz
... skip ...
After seeing various ext2 related things, I realised it was probably a problem
with something else. Reviewing grub's menu.lst, it becomes obvious:
title Debian GNU/Linux, kernel 2.6.18-5-686
root (hd0,0)
kernel /boot/vmlinuz-2.6.18-5-686 root=/dev/hda1 ro
initrd /boot/initrd.img-2.6.18-5-686
savedefault
.... skip...
title MikroTik 2.4.31 / 2.9.46
root (hd0,0)
kernel /boot/vmlinuz-2.4.31
initrd /boot/mikro-initrd.rgz
The kernel line for the MikroTik entry misses out on some parameters. Fixing
the applicable line, (kernel /boot/vmlinuz-2.4.31
root=/dev/hda1 ro single) and rebooting, it works. At least the decompressing
of the kernel image can come in use for further investigation work. One last
thing to note is that it requires the debian modutils package to work with 2.4
kernels.
Anyways, moving on, the debian image boots up and works reasonably with their
kernel / modules. I had to load the ne2k-pci module manually (via modprobe
ne2k-pci) to bring up networking under QEMU.
Another issue I had under the debian/mikrotik hybrid I created, was that the
MikroTik kernel does not have AF_UNIX/AF_FILE support built-in, so useful programs like
sysklogd and sshd would not run by default… However, it ships this as a
module, so modprobe unix took care of this issue.
In order to run the MikroTik binaries (/nova/bin/), I needed to copy various
files. I copied /nova/ over, and made a directory called /lib_mikro/ where I
could copy various libary files over that resided in the /lib directory on
the MikroTik installation.
In order to use these libraries in a non-standard directory location, the
environment variable LD_LIBRARY_PATH can be set. This way only the
applicable MikroTik binaries can be ran with correct library versions.
Examining the fileman binary
Doing some prelimiary analysis on the fileman binary shows that it appears
to be expecting a network file descriptor on fd 3.
[box] $ LD_LIBRARY_PATH=/lib_mikro/ strace -f ./fileman
execve("./fileman", ["./fileman"], [/* 14 vars */]) = 0
uname({sys="Linux", node="debian", ...}) = 0
brk(0) = 0x805861c
...
rt_sigaction(SIGFPE, {0x4002c8ca, [], SA_RESTORER|SA_RESTART|SA_SIGINFO,
0x4009d4c0}, NULL, 8) = 0
getsockname(3, 0xbffffd18, [110]) = -1 EBADF (Bad file descriptor)
socket(PF_FILE, SOCK_STREAM, 0) = -1 EAFNOSUPPORT (Address family not
supported by protocol)
exit_group(1) = ?
Process 2147 detached
Unfortunately, debian's version of bash does not have /dev/tcp/ support,
so unfortunately it's not as easy as nc -l 12121 on one terminal, and …
./fileman 3</dev/tcp/blah/.
After quickly writing some code to do what's needed, we can run fileman (and
probably others). It's available here if you would like it. Usage
is simple, python fd3.py <program to execute> <arguments for program>. Note
that the first argument you specify for the program is argv[0] - not argv[1].
An example of a command line would be python fd3.py /usr/bin/strace strace -f
/path/to/program.
Another issue appeared when trying to run fileman with a valid file descriptor
on fd 3 - it wanted /tmp/novasock to be a valid file descriptor:
[box] $ LD_LIBRARY_PATH=/lib_mikro fd3 `which strace` strace -f ./fileman
getsockname(3, {sa_family=AF_INET, sin_port=htons(31313), sin_addr=inet_addr("192.168.254.3")}, [16]) = 0
socket(PF_FILE, SOCK_STREAM, 0) = 4
connect(4, {sa_family=AF_FILE, path="/tmp/novasock"}, 110) = -1 ENOENT (No such file or directory)
close(4) = 0
exit_group(1) = ?
Looking for /tmp/novasock we find the the loader binary seems to have what
we're after:
[box] $ strings -f * | grep novasock
loader: /tmp/novasock
Analysing kernel module and userland interaction
Performing some initial analysis via strace on loader reveals an interesting /
startling behaviour in loader:
[box] $ LD_LIBRARY_PATH=/lib_mikro strace -f ./loader
...
[pid 2361] create_module("qwink", 1430) = 0xc8831000
[pid 2361] init_module(0x80553fc, 134587376, umovestr: Input/output error
0xa7) = 0
[pid 2361] delete_module("qwink") = 0
...
The interesting thing about this particular piece of code is that it is
loading a linux kernel module (LKM), and immediately removes the kernel
module. This is particularly interesting as it would appear to be a kernel
module that's meant to be out of sight.
To dump the module, we could hook init_module to perform our required actions,
however, first we have to verify it's easily possible:
[box[ $ objdump -R loader | grep -i module
[box] $
This is interesting, as it appears to not be importing the various module
library calls, performing some more analysis:
[box] $ objdump -dtrs loader | grep module
...
804fb69: e8 3a 42 00 00 call 8053da8 <delete_module+0x336>
08053a20 <create_module>:
8053a36: 76 0c jbe 8053a44 <create_module+0x24>
08053a49 <init_module>:
...
The interesting thing about this output is the sections where the module names
are surrounded by the angle brackets; this indicates that those functions
exist in the .text of loader:
08053a49 <init_module>:
8053a49: 55 push %ebp
8053a4a: b8 80 00 00 00 mov $0x80,%eax
8053a4f: 89 e5 mov %esp,%ebp
8053a51: 53 push %ebx
8053a52: 8b 4d 0c mov 0xc(%ebp),%ecx
8053a55: 8b 5d 08 mov 0x8(%ebp),%ebx
8053a58: cd 80 int $0x80
8053a5a: 83 f8 82 cmp $0xffffff82,%eax
8053a5d: 89 c3 mov %eax,%ebx
8053a5f: 76 0c jbe 8053a6d <init_module+0x24>
8053a61: f7 db neg %ebx
8053a63: e8 54 70 ff ff call 804aabc <__errno_location@plt>
8053a68: 89 18 mov %ebx,(%eax)
8053a6a: 83 cb ff or $0xffffffff,%ebx
8053a6d: 89 d8 mov %ebx,%eax
8053a6f: 5b pop %ebx
8053a70: 5d pop %ebp
8053a71: c3 ret
This appears to be a standard implementation of the _syscallX() macros in the
asm/unistd.h. While we're staring at objdump -d output, we may as well
look at where this code is (statically) being called from:
8053ea6: ff 75 dc pushl 0xffffffdc(%ebp)
8053ea9: ff 35 20 64 05 08 pushl 0x8056420
8053eaf: e8 95 fb ff ff call 8053a49 <init_module>
At 0x8053ea9 it pushes a static address (0x8056420) which does not
correspond to our strace output. Having a brief look at where that variable
is used before hand:
[box] $ objdump -dtrsRS loader | grep -C 2 8056420
...
8053db4: 83 ec 18 sub $0x18,%esp
8053db7: c7 45 f0 00 00 00 00 movl $0x0,0xfffffff0(%ebp)
8053dbe: 8b 3d 20 64 05 08 mov 0x8056420,%edi
8053dc4: ba 10 00 00 00 mov $0x10,%edx
8053dc9: f2 ae repnz scas %es:(%edi),%al
--
8053e19: e8 3e 70 ff ff call 804ae5c <memcpy@plt>
8053e1e: 53 push %ebx
8053e1f: ff 35 20 64 05 08 pushl 0x8056420
8053e25: 56 push %esi
8053e26: e8 31 70 ff ff call 804ae5c <memcpy@plt>
8053e2b: ff 75 ec pushl 0xffffffec(%ebp)
8053e2e: ff 35 20 64 05 08 pushl 0x8056420
8053e34: e8 e7 fb ff ff call 8053a20 <create_module>
8053e39: 83 c4 24 add $0x24,%esp
--
8053ea1: e8 57 fd ff ff call 8053bfd <delete_module+0x18b>
8053ea6: ff 75 dc pushl 0xffffffdc(%ebp)
8053ea9: ff 35 20 64 05 08 pushl 0x8056420
8053eaf: e8 95 fb ff ff call 8053a49 <init_module>
8053eb4: 83 c4 10 add $0x10,%esp
--
8053ee0: 83 c4 0c add $0xc,%esp
8053ee3: 8b 55 f0 mov 0xfffffff0(%ebp),%edx
8053ee6: ff 35 20 64 05 08 pushl 0x8056420
8053eec: 39 c2 cmp %eax,%edx
8053eee: 0f 94 c3 sete %bl
So, it appears it's used a bit to set it up. Being the somewhat lazy type, it
would be easy enough to write a dynamic library to modify the .text segment
to insert a hook. The mechanisms used for this is discussed in a paper I wrote
available
here.
The hook code aim is simple, which is to dump the applicable information sent
to init_module. It also appears that the init_module function declaration
changes between 2.4 and 2.6 kernel versions:
2.4:
*`int init_module(const char *name, struct module *image);`
.2.6:
* `long sys_init_module (void *umod, unsigned long len, const char *uargs);`
The strace output above is for 2.6, not 2.4. After locating a suitable
2.4 init_module man page, we see that the second parameter is a pointer to a
structure:
The module image begins with a module structure and is followed by code and data as appropriate. The module structure is defined as follows:
struct module {
unsigned long size_of_struct;
struct module *next;
const char *name;
unsigned long size;
long usecount;
unsigned long flags;
unsigned int nsyms;
unsigned int ndeps;
struct module_symbol *syms;
struct module_ref *deps;
struct module_ref *refs;
int (*init)(void);
void (*cleanup)(void);
const struct exception_table_entry *ex_table_start;
const struct exception_table_entry *ex_table_end;
#ifdef __alpha__
unsigned long gp;
#endif
};
At least the required information is available to make it easier. After
writing the required the code (which is available
here)
Running our hooking library code (which is available , we get:
[box] $ LD_LIBRARY_PATH=/lib_mikro LD_PRELOAD=/tmp/hook-loader.so ./loader
forked
creating loader
--> In an int3 handler
--> Create module return address is 0xc8831000
--> In an int3 handler
--> working our magic for qwink
Matching the dumped header information with the struct module output, we get:
[box] $ xxd /tmp/module_header
0000000: 3c00 0000 0000 0000 9015 83c8 9605 0000 <...............
0000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
0000020: 0000 0000 0000 0000 0000 0000 4010 83c8 ............@...
0000030: 0000 0000 0000 0000 0000 0000 ............
... matching up with the above struct module ...
size_of_struct: 0x0000003c (0x0000)
next: NULL (0x0004)
name: 0xc8831590 (0x0008)
size: 0x00000596 (0x000c)
usecount: NULL (0x0010)
flags: 0 (0x0014)
nsyms: 0 (0x0018)
ndeps: 0 (0x001c)
syms: NULL (0x0020)
deps: NULL (0x0024)
refs: NULL (0x0028)
init: 0xc8831040 (0x002c)
cleanup: NULL (0x0030)
ex_table_start: NULL (0x0034)
ex_table_end: NULL (0x0038)
So, our output in /tmp/module_text starts at 0xc883103c, and is 1430 bytes
long. Disassembling the binary dump in /tmp/module_text, looking for the
init function pointer that gets called:
[box] $ ndisasm -b 32 -o $((0xc883103c)) /tmp/module_text
...
C8831040 83EC18 sub esp,byte +0x18
C8831043 53 push ebx
C8831044 E8F7040000 call 0xc8831540
C8831049 8D5818 lea ebx,[eax+0x18]
C883104C C7430400000000 mov dword [ebx+0x4],0x0
C8831053 C7401800000000 mov dword [eax+0x18],0x0
...
We can combine the gdb debugging functionality in qemu in conjunction with a
disassembler such as IDA Pro , we can use the gdb
stub functionality of qemu, we can disassemble and follow what is happening
when the init code is executed.
Testing our analysis against the init function by setting break points on:
Just to make sure we catch everything.. I don't expect 0xc053ffd0 to be hit,
but it is a valid kernel address :)
Testing the theory out under gdb/qemu doesn't pan out how it was expected to be:
(gdb) break *0xc053ffd0
Breakpoint 4 at 0xc053ffd0
(gdb) break *0xc8831040
Breakpoint 5 at 0xc8831040
(gdb) c
Continuing.
... Run the loader program ...
Breakpoint 5, 0xc8831040 in ?? ()
(gdb) x/10i $eip
0xc8831040: sub $0x18,%esp
0xc8831043: push %ebx
0xc8831044: call 0xc8831540
0xc8831049: lea 0x18(%eax),%ebx
0xc883104c: movl $0x0,0x4(%ebx)
0xc8831053: movl $0x0,0x18(%eax)
0xc883105a: mov 0xc0252024,%eax
0xc883105f: inc %eax
0xc8831060: mov %eax,0x8(%ebx)
0xc8831063: call 0xc8831090
(gdb) i r ebx
ebx 0xc8831000 -930934784
Now that we're able to trace this code, we should work out what information
would be desirable to make this process far easier:
-
Ideally, I'd love a complete, working System.map, however, looking at the
files available, it doesn't appear to be available. Scratching that one
off, we get:
-
Use symbols provided by /proc/ksyms to help our reversing effort. That's a
useful start, but provides a limited view of what is available.
Another couple of things we could try to help our reversing efforts:
-
Compile a custom kernel similar to theres as possible, with same compiler /
distro. Once it's been compiled, we can extract
function signatures from the
vmlinux file, and attempt to apply it to the extracted image.
[box] $ cat /proc/version
Linux version 2.4.31 (build@builder2) (gcc version 2.95.4 20011002 (Debian prerelease)) #1 Fri Aug 10 12:43:55 EEST 2007
After some researching, it was determined that the gcc version 2.95.4
20011002
is available if a default install of Debian 3.0r1 is installed. I only needed
cd1 and cd2 from the set of cd's to install the required tools.
After performing a compile of the default kernel configuration shipped in
2.4.31 sans PCMCIA support, I used the 2pelf tool available
here to generate the signatures off
the .o files in the linux tree, then used the IDA tool sigmake to generate
the signature information. (I love bash scripting to automate these tasks).
If needed, the signatures could be regenerated several times with different
compilation parameters in order to increase success of signature matching.
Since it would be helpful to have a full kernel text image in
IDA Pro, we can use the memsave function in QEMU
to generate an appropriate memory dump. To get the most useful memory dump,
I'll generate the dump when qemu has hit the breakpoint on the "qwink" init
function.
The vmlinux file generated when I compiled the 2.4.31 kernel indicated that
the kernel got loaded at 0xc0100000, so we'll dump from there to
0xd0000000… This actually turned out to be a bad idea, as IDA couldn't
analyse such large files. After some experimentation and looking at the output
of objdump -fp on the compiled vmlinux file, I dumped from 0xc0100000 to
0xc1000000.
(qemu) memsave 3222274048 15728640 d
*pause*
(qemu)
The first number is 0xc0100000 and the second is 0xc1000000 - 0xc0100000.
After dumping the kernel memory to disk, then loading it into IDA, and
applying the signatures that was generated before, we see a lot of function
names pop up. Sometimes they're correct, sometimes they don't appear to be,
sometimes they don't find functions we find interesting, but in general they
save us a lot of work :)
The init code is as follows from IDA:
seg001:C8831040 module_init_code:
seg001:C8831040 sub esp, 18h
seg001:C8831043 push ebx
seg001:C8831044 call GET_C8831550 ; eax = 0xc8831550
seg001:C8831049 lea ebx, [eax+18h] ; ebx = eax + 0x18 = 0xc8831568, this is struct timer_list data
seg001:C883104C mov dword ptr [ebx+4], 0 ; sets next pointer to null
seg001:C8831053 mov dword ptr [eax+18h], 0 ; sets prev pointer to null ?
seg001:C883105A mov eax, ds:jiffies
seg001:C883105F inc eax ; schedules the code to be executed in one jiffie
seg001:C8831060 mov [ebx+8], eax ; sets the scheduler expires
seg001:C8831063 call GET_C88310a0
seg001:C8831068 mov [ebx+10h], eax ; function pointer used (execd_by_timer) below
seg001:C883106B mov eax, 0FFFFE000h
seg001:C8831070 and eax, esp ; get current
seg001:C8831072 mov [ebx+0Ch], eax ; data pointer for call back. current task
seg001:C8831075 add esp, -0Ch ; no idea what it's doing here. possibly aligning stack to paragraph boundary?
seg001:C8831078 mov eax, offset add_timer
seg001:C883107D push ebx ; push offset to struct timer_list data
seg001:C883107E call eax ; add_timer
seg001:C8831080 xor eax, eax ; return NULL
seg001:C8831082 add esp, 10h
seg001:C8831085 pop ebx
seg001:C8831086 add esp, 18h
seg001:C8831089 retn
-
I discovered the mov eax, offset 0xc011b170 / call eax was the function
add_timer based on the string "bug: kernel added timer twice at %p.\n"
and then grep'ing the kernel source tree for added timer twice at,
which lead me to kernel/timer.c.
-
I discovered 0xc0252024 was the jiffies variable by cross referencing
where that variable was used, and looking for some identifable
charactistics. I found what I was looking with a little bit of reverse
engineering, and cross referencing what was found with the linux kernel
code that I had available to identify the variable name.
-
Cross referencing structures once they were discovered helped things a lot.
(For example, the struct timer).
-
After following through the code that would be executed by the timer code,
it appeared to be generating a rc4 key, and then generating a 4 byte value,
and then calling access_process_vm to write 4 bytes based on the values
passed in. Luckily the function signatures generated earlier identified the
access_process_vm function for me, and saved a fair amount of effort.
-
QEMU's breakpoints come in use for following this in gdb.
Having a look at the IDA loader binary, we see:
Example: Loader: load_module_into_kernel
.text:08053DA8 load_module_into_kernel proc near ; CODE XREF: sub_804DD20+25p
.text:08053DA8 ; sub_804F572+10Dp ...
.text:08053DA8
.text:08053DA8 constructed_image= dword ptr -24h
.text:08053DA8 roundup_size = dword ptr -20h
.text:08053DA8 start_of_module_data= dword ptr -18h
.text:08053DA8 total_size = dword ptr -14h
.text:08053DA8 kernel_modified_variable= dword ptr -10h
.text:08053DA8 var_C = dword ptr -0Ch
.text:08053DA8 tv_usecs_challenge= dword ptr 8
.text:08053DA8
.text:08053DA8 push ebp
.text:08053DA9 cld ; clear direction flag
.text:08053DAA mov ebp, esp
.text:08053DAC push edi
.text:08053DAD xor eax, eax
.text:08053DAF push esi
.text:08053DB0 or ecx, 0FFFFFFFFh
.text:08053DB3 push ebx
.text:08053DB4 sub esp, 18h
.text:08053DB7 mov [ebp+kernel_modified_variable], 0 ; initialise the variable written to by access_process_vm to 0
.text:08053DBE mov edi, module_name
.text:08053DC4 mov edx, 10h
.text:08053DC9 repne scasb
.text:08053DCB mov ebx, ecx
.text:08053DCD mov eax, 3Ch
.text:08053DD2 not ebx ; embedded strlen()
.text:08053DD4 call sub_8053D95 ; calcuate if additional space is needed in a memory copy later on
.text:08053DD9 mov [ebp+roundup_size], eax ; save the calcutation for later use
.text:08053DDC lea edx, [eax+ebx+1360] ; calcuate total size needed, + 1360
.text:08053DE3 push edx ; size
.text:08053DE4 mov [ebp+total_size], edx
.text:08053DE7 call _malloc ; allocate the required space
.text:08053DEC mov [ebp+constructed_image], eax ; save the results of malloc()
.text:08053DEF mov ecx, [ebp+total_size]
.text:08053DF2 cld ; clear direction flag
.text:08053DF3 mov edi, eax
.text:08053DF5 xor eax, eax ; write nulls
.text:08053DF7 rep stosb ; embedded memset
.text:08053DF9 push 510h ; size_t
.text:08053DFE mov edi, [ebp+constructed_image] ; get the allocated memory
.text:08053E01 add edi, [ebp+roundup_size] ; move past the module structure
.text:08053E04 push offset module_init_code ; void *
.text:08053E09 lea edx, [edi+510h] ; edx now points PAST the module_init_code and all that
.text:08053E0F push edi ; void *
.text:08053E10 lea esi, [edi+1360] ; esi = module name pointer
.text:08053E16 mov [ebp+start_of_module_data], edx
.text:08053E19 call _memcpy ; copy the module_init_code (of length 0x510)
.text:08053E19 ; to the allocated memory
.text:08053E1E push ebx ; size_t. this is calcuated previously from strlen(module_name)
.text:08053E1F push module_name ; void *
.text:08053E25 push esi ; points to module name
.text:08053E26 call _memcpy ; setup module name
.text:08053E2B push [ebp+total_size] ; int
.text:08053E2E push module_name ; name
.text:08053E34 call create_module
.text:08053E39 add esp, 24h
.text:08053E3C cmp eax, 0FFFFFFFFh ; eax contains base of the allocated kernel memory
.text:08053E3F jnz short loc_8053E48
.text:08053E41 push offset aFailedToCreate ; "failed to create module"
.text:08053E46 jmp short loc_8053EC0
.text:08053E48 ; ---------------------------------------------------------------------------
.text:08053E48
.text:08053E48 loc_8053E48: ; CODE XREF: load_module_into_kernel+97j
.text:08053E48 mov ecx, [ebp+constructed_image]
.text:08053E4B mov dword ptr [ecx], 3Ch ; set size of module header to 0x3c
.text:08053E51 mov ecx, [ebp+roundup_size]
.text:08053E54 lea edx, [eax+ecx] ; edx = start of module code, in kernel space
.text:08053E57 mov ecx, [ebp+constructed_image]
.text:08053E5A lea eax, [edx+1360] ; eax = end of module code, start of "qwink"
.text:08053E60 mov [ecx+2Ch], edx ; set module size
.text:08053E63 mov edx, [ebp+tv_usecs_challenge] ; arg_0 = tv.tv_usecs ?
.text:08053E66 mov [ecx+8], eax ; write the name pointer, points to end of module, "qwink"
.text:08053E69 mov eax, [ebp+total_size]
.text:08053E6C mov [edi+510h], edx ; write challenge to kernel image (data section)
.text:08053E72 mov [ecx+0Ch], eax ; write size of complete module to header
.text:08053E75 mov ecx, [ebp+start_of_module_data]
.text:08053E78 mov eax, kernel_ptr_c0105000
.text:08053E7D mov [ecx+4], eax
.text:08053E80 mov eax, dword_8056A68 ; some mystical value
.text:08053E85 mov [ecx+8], eax
.text:08053E88 mov eax, dword_8056A6C ; 0x3f
.text:08053E8D mov [ecx+0Ch], eax
.text:08053E90 lea eax, [ebp+kernel_modified_variable] ; get the address of the variable
.text:08053E93 mov [ecx+10h], eax ; variable that is going to be written to
.text:08053E96 call alloc_rc4_t
.text:08053E9B push [ebp+start_of_module_data]
.text:08053E9E mov esi, eax
.text:08053EA0 push eax
.text:08053EA1 call rc4_init_key_encrypt ; (rc4_t, start_of_module_data)
.text:08053EA6 push [ebp+constructed_image] ; image
.text:08053EA9 push module_name ; name
.text:08053EAF call init_module
.text:08053EB4 add esp, 10h
.text:08053EB7 test eax, eax
.text:08053EB9 jz short loc_8053EC9
.text:08053EBB push offset aFailedToLoadMo ; "failed to load module"
.text:08053EC0
.text:08053EC0 loc_8053EC0: ; CODE XREF: load_module_into_kernel+9Ej
.text:08053EC0 call _puts
.text:08053EC5 xor eax, eax
.text:08053EC7 jmp short loc_8053EFE
.text:08053EC9 ; ---------------------------------------------------------------------------
.text:08053EC9
.text:08053EC9 loc_8053EC9: ; CODE XREF: load_module_into_kernel+111j
.text:08053EC9 ; load_module_into_kernel+126_j
.text:08053EC9 mov eax, [ebp+kernel_modified_variable]
.text:08053ECC test eax, eax
.text:08053ECE jz short loc_8053EC9 ; while the variable hasn't been modified, loop. Ugh.
.text:08053ED0 push offset unk_8056960 ; not sure yet
.text:08053ED5 xor ebx, ebx
.text:08053ED7 push [ebp+tv_usecs_challenge] ; challenge
.text:08053EDA push esi ; rc4 structure, returned from rc4_init
.text:08053EDB call check_challenge_response
.text:08053EE0 add esp, 0Ch
.text:08053EE3 mov edx, [ebp+kernel_modified_variable]
.text:08053EE6 push module_name ; name
.text:08053EEC cmp edx, eax ; result from check_challenge_response
.text:08053EEE setz bl ; set return code if they're the same
.text:08053EF1 call delete_module
.text:08053EF6 push esi
.text:08053EF7 call free_rc4_t_tailcall
.text:08053EFC mov eax, ebx ; set return value (based on check_challenge_response)
.text:08053EFE
.text:08053EFE loc_8053EFE: ; CODE XREF: load_module_into_kernel+11Fj
.text:08053EFE lea esp, [ebp-0Ch]
.text:08053F01 pop ebx
.text:08053F02 pop esi
.text:08053F03 pop edi
.text:08053F04 pop ebp
.text:08053F05 retn
.text:08053F05 load_module_into_kernel endp
.text:08053F05
Example: Loader: check_challenge_response
.text:08053CEA check_challenge_response proc near ; CODE XREF: load_module_into_kernel+133p
.text:08053CEA
.text:08053CEA local_rc4_structure= dword ptr -14h
.text:08053CEA challenge_copy = dword ptr -10h
.text:08053CEA var_C = dword ptr -0Ch
.text:08053CEA rc4_t = dword ptr 8
.text:08053CEA challenge = dword ptr 0Ch
.text:08053CEA unknown_data = dword ptr 10h
.text:08053CEA
.text:08053CEA push ebp
.text:08053CEB mov ebp, esp
.text:08053CED push edi
.text:08053CEE push esi
.text:08053CEF xor esi, esi
.text:08053CF1 push ebx
.text:08053CF2 push ecx
.text:08053CF3 push ecx
.text:08053CF4 mov edi, [ebp+rc4_t]
.text:08053CF7 mov eax, [ebp+challenge]
.text:08053CFA push edi
.text:08053CFB mov [ebp+challenge_copy], eax
.text:08053CFE call rc4_dup_struct
.text:08053D03 mov [ebp+local_rc4_structure], eax ; duplicated structure
.text:08053D06 mov eax, [ebp+unknown_data]
.text:08053D09 push dword ptr [eax+100h] ; offset in 256 bytes
.text:08053D0F push edi ; rc4 structure passed in
.text:08053D10 call rc4_set_key
.text:08053D15 add esp, 0Ch
.text:08053D18
.text:08053D18 loc_8053D18: ; CODE XREF: check_challenge_response+91j
.text:08053D18 push edi
.text:08053D19 call rc4_get_byte
.text:08053D1E mov edx, [ebp+local_rc4_structure]
.text:08053D21 mov bl, al ; gets a byte from the rc4 structure which was passed in (and initialised later on)
.text:08053D23 mov ecx, esi ; esi is a loop counter
.text:08053D25 and ecx, 3 ; index into key material
.text:08053D28 movzx eax, byte ptr [edx+esi] ; index into the duplicated structure
.text:08053D2C mov edx, [ebp+unknown_data]
.text:08053D2F movzx eax, byte ptr [edx+eax] ; get a byte of the unknown data
.text:08053D33 pop edx ; edx = local rc4 structure
.text:08053D34 mov edx, ebx ; get the byte that was generated via rc4_get_byte
.text:08053D36 and edx, 3 ; perform a switch on the last 4 bytes
.text:08053D39 cmp edx, 1
.text:08053D3C jz short loc_8053D5A
.text:08053D3E jg short loc_8053D46
.text:08053D40 test edx, edx
.text:08053D42 jz short loc_8053D52
.text:08053D44 jmp short loc_8053D74
.text:08053D46 ; ---------------------------------------------------------------------------
.text:08053D46
.text:08053D46 loc_8053D46: ; CODE XREF: check_challenge_response+54j
.text:08053D46 cmp edx, 2
.text:08053D49 jz short loc_8053D62
.text:08053D4B cmp edx, 3
.text:08053D4E jz short loc_8053D6A
.text:08053D50 jmp short loc_8053D74
.text:08053D52 ; ---------------------------------------------------------------------------
.text:08053D52
.text:08053D52 loc_8053D52: ; CODE XREF: check_challenge_response+58j
.text:08053D52 xor al, bl
.text:08053D54 add byte ptr [ebp+ecx+challenge_copy], al
.text:08053D58 jmp short loc_8053D74
.text:08053D5A ; ---------------------------------------------------------------------------
.text:08053D5A
.text:08053D5A loc_8053D5A: ; CODE XREF: check_challenge_response+52j
.text:08053D5A add al, bl
.text:08053D5C xor byte ptr [ebp+ecx+challenge_copy], al
.text:08053D60 jmp short loc_8053D74
.text:08053D62 ; ---------------------------------------------------------------------------
.text:08053D62
.text:08053D62 loc_8053D62: ; CODE XREF: check_challenge_response+5Fj
.text:08053D62 xor bl, byte ptr [ebp+ecx+challenge_copy]
.text:08053D66 add al, bl
.text:08053D68 jmp short loc_8053D70
.text:08053D6A ; ---------------------------------------------------------------------------
.text:08053D6A
.text:08053D6A loc_8053D6A: ; CODE XREF: check_challenge_response+64j
.text:08053D6A add al, byte ptr [ebp+ecx+challenge_copy]
.text:08053D6E xor al, bl
.text:08053D70
.text:08053D70 loc_8053D70: ; CODE XREF: check_challenge_response+7Ej
.text:08053D70 mov byte ptr [ebp+ecx+challenge_copy], al
.text:08053D74
.text:08053D74 loc_8053D74: ; CODE XREF: check_challenge_response+5Aj
.text:08053D74 ; check_challenge_response+66j ...
.text:08053D74 inc esi
.text:08053D75 cmp esi, 100h
.text:08053D7B jnz short loc_8053D18
.text:08053D7D push [ebp+local_rc4_structure]
.text:08053D80 call free_rc4_t_tailcall
.text:08053D85 mov eax, [ebp+challenge_copy]
.text:08053D88 lea esp, [ebp-0Ch]
.text:08053D8B pop ebx
.text:08053D8C or eax, 80000000h ; ret |= 0x80000000l
.text:08053D91 pop esi
.text:08053D92 pop edi
.text:08053D93 pop ebp
.text:08053D94 retn
.text:08053D94 check_challenge_response endp
.text:08053D94
.text:08053D95
.text:08053D95 ; ??????????????? S U B R O U T I N E ???????????????????????????????????????
.text:08053D95
.text:08053D95 ; Attributes: bp-based frame
.text:08053D95
.text:08053D95 sub_8053D95 proc near ; CODE XREF: load_module_into_kernel+2Cp
.text:08053D95 push ebp
.text:08053D96 dec edx ; 16 -> 15
.text:08053D97 test eax, edx ; eax = 0x3c
.text:08053D99 mov ebp, esp
.text:08053D9B mov ecx, eax
.text:08053D9D jz short loc_8053DA4
.text:08053D9F or edx, eax
.text:08053DA1 lea ecx, [edx+1]
.text:08053DA4
.text:08053DA4 loc_8053DA4: ; CODE XREF: sub_8053D95+8j
.text:08053DA4 pop ebp
.text:08053DA5 mov eax, ecx
.text:08053DA7 retn
.text:08053DA7 sub_8053D95 endp
.text:08053DA7
.text:08053DA8
.text:08053DA8 ; ??????????????? S U B R O U T I N E ???????????????????????????????????????
.text:08053DA8
.text:08053DA8 ; Attributes: bp-based frame
.text:08053DA8
Example: Loader: rc4_init_key_encrypt
.text:08053BFD rc4_init_key_encrypt proc near ; CODE XREF: load_module_into_kernel+F9p
.text:08053BFD
.text:08053BFD var_8 = dword ptr -8
.text:08053BFD rc4_t = dword ptr 8
.text:08053BFD start_of_module_data= dword ptr 0Ch
.text:08053BFD
.text:08053BFD push ebp
.text:08053BFE mov ebp, esp
.text:08053C00 push esi
.text:08053C01 mov esi, [ebp+rc4_t]
.text:08053C04 push ebx
.text:08053C05 mov ebx, [ebp+start_of_module_data]
.text:08053C08 push esi
.text:08053C09 call init_rc4_t
.text:08053C0E push dword ptr [ebx] ; push the encryption key
.text:08053C10 add ebx, 4 ; move the data along
.text:08053C13 push esi ; push the rc4_t context
.text:08053C14 call rc4_set_key
.text:08053C19 push 1000 ; length
.text:08053C1E push esi ; rc4 structure
.text:08053C1F call rc4_prevent_weak_bytes
.text:08053C24 push 10h ; size
.text:08053C26 push ebx ; data
.text:08053C27 push esi ; rc4 structure
.text:08053C28 call rc4_crypt ; encrypts 16 bytes that go to the kernel module
.text:08053C2D lea esp, [ebp-8]
.text:08053C30 pop ebx
.text:08053C31 pop esi
.text:08053C32 pop ebp
.text:08053C33 retn
.text:08053C33 rc4_init_key_encrypt endp
In general, the feeling is that the loader binary tries to ensure it's running
on the same kernel the module was written for, because it is using hard coded
offsets to the kernel data. I was disappointed that this module the loader
binary inserted didn't appear to be malicious.
To bypass this somewhat artifical restriction the binary imposes is somewhat
easily done. The previous code written to hook the create_module and
init_module code, can be modified to:
to avoid this restriction. I haven't tested this, but it should work :p
Continuing the analysis of loader
Before we can continue the analysis of the fileman binary, we still need to
get the loader binary running. Running a strace reveals the current issue
with loader:
[box] # LD_LIBRARY_PATH=/lib_mikro strace -f ./loader
execve("./loader", ["./loader"], [/* 15 vars */]) = 0
uname({sys="Linux", node="debian", ...}) = 0
...
[pid 1727] rt_sigaction(SIGSEGV, {0x4002c8ca, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x400774c0}, NULL, 8) = 0
[pid 1727] rt_sigaction(SIGILL, {0x4002c8ca, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x400774c0}, NULL, 8) = 0
[pid 1727] rt_sigaction(SIGABRT, {0x4002c8ca, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x400774c0}, NULL, 8) = 0
[pid 1727] rt_sigaction(SIGBUS, {0x4002c8ca, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x400774c0}, NULL, 8) = 0
[pid 1727] rt_sigaction(SIGFPE, {0x4002c8ca, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x400774c0}, NULL, 8) = 0
[pid 1727] gettimeofday({1189098093, 745332}, NULL) = 0
[pid 1727] create_module("qwink", 1430) = 0xc8833000
[pid 1727] init_module(0x80553fc, 134587376, umovestr: Input/output error
0x7c) = 0
[pid 1726] waitpid(1727, Process 1726 suspended
<unfinished ...>
[pid 1727] delete_module("qwink") = 0
...
[pid 1727] open("/dev/panics", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 1727] open("/proc/cmdline", O_RDONLY) = 3
[pid 1727] fstat64(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
[pid 1727] mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40170000
[pid 1727] read(3, "root=/dev/hda1 ro single\n", 4096) = 25
[pid 1727] close(3) = 0
[pid 1727] munmap(0x40170000, 4096) = 0
[pid 1727] exit_group(1) = ?
It appears some functionality that is provided elsewhere needs to be
initialised first.
Having a look in IDA at loader (cross referencing on /proc/cmdline),
indicates that the /proc/cmdline must contain MBR= followed by
a pattern that matches %x.
.text:0804DBD4 push offset name ; "/proc/cmdline"
.text:0804DBD9 call _fopen ; open the file, read only
.text:0804DBDE pop ebx
.text:0804DBDF mov esi, eax
.text:0804DBE1 test esi, esi
.text:0804DBE3 pop eax
.text:0804DBE4 jz short loc_804DC34 ; if opening the file failed, jump down to exit(1)
.text:0804DBE6 push esi ; FILE *
.text:0804DBE7 lea ebx, [ebp+var_408]
.text:0804DBED push 1024 ; int
.text:0804DBF2 push ebx ; char *
.text:0804DBF3 call _fgets ; read in 0x400 bytes
.text:0804DBF8 push esi ; FILE *
.text:0804DBF9 call _fclose
.text:0804DBFE push offset aMbr ; "MBR="
.text:0804DC03 push ebx ; char *
.text:0804DC04 call _strstr ; look for MBR= in the string
.text:0804DC09 add esp, 18h
.text:0804DC0C test eax, eax
.text:0804DC0E mov edx, eax
.text:0804DC10 jz short loc_804DC34 ; don't find it, exit out
.text:0804DC12 mov [ebp+var_40C], 0 ; initialse the number read in
.text:0804DC1C lea eax, [ebp+var_40C] ; load the address in
.text:0804DC22 push eax
.text:0804DC23 push offset aMbrX ; "MBR=%x"
.text:0804DC28 push edx ; char *
.text:0804DC29 call _sscanf ; parse the string
.text:0804DC2E add esp, 0Ch
.text:0804DC31 dec eax
.text:0804DC32 jz short loc_804DC3B
.text:0804DC34
.text:0804DC34 loc_804DC34: ; CODE XREF: sub_804DBC4+20j
.text:0804DC34 ; sub_804DBC4+4Cj
.text:0804DC34 push 1 ; status
.text:0804DC36 call _exit
.text:0804DC3B ; ---------------------------------------------
Looking up information on
Master boot records (MBR),
indicates that the MBR starts at 0x1BE. However, looking where the code is
called, there is a comparision to see if it's above a certain size. Restarting
with an updated kernel line with MBR=0, lets the binary run, and listen on a
network socket.
Summary
This article has shown usage QEMU with GDB to debug Linux kernel modules,
along with static disassembly provided by IDA Pro. It has covered analysing an
obscured kernel module that was bound tightly to a specific, vendor, kernel.
(started 22/8/2007, added 11/10/2007)
The MikroTik Wireless Router is a
Linux embedded wireless router, focusing on various functionality such as
bandwidth management, Firewalling, VPN server/client, and various other
things. As with all embedded linux based software, it is interesting to pull
it apart :)
It has been around for a while now… a couple of years ago when I analysed
the software / pulling it apart, it had drivers/firmware to turn standard Orinoco
wireless cards into an Access Point (which as far as I know isn't possible
otherwise, at least not when I was looking at it.)
For the purposes of this article, I am looking at mikrotik-2.9.46.iso
(MD5sum: 65aa908dd748ccf72ad9f588613dfe31, SHA1sum:
5e5ed13498db8d9745a701f75e58da3ef6701e58). For the most part, I have used
QEMU to emulate the hardware/software
environment to install it on. This has several advantages, such as being able
to edit the "disk" it's using easily, amongst other things.
Initial observations
After installing MikroTik, strings was ran on the resulting disk image, to see
what could be initially noticed.
Inconsistent / random GCC versions used
One of the most amusing things that was noticed, was the various GCC version
strings that were in the output:
box $ strings -a mikrotik.img | grep ^GCC: | sort | uniq
GCC: (GNU) 2.7.2.3
GCC: (GNU) 2.95.4 20011002 (Debian prerelease)
GCC: (GNU) 3.0.4
GCC: (GNU) 3.4.6 (Debian 3.4.6-5)
GCC: (GNU) 4.0.3 (Debian 4.0.3-1)
GCC: (GNU) egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)
Observations
-
They don't strip debugging information from binaries / libraries, which
usually is a good idea when you're concerned about disk usage (more so for
embedded systems).
-
The differing GCC versions seem to indicate that they haven't ran a make
clean on the source code in ages, or they have lost the source code. Doesn't
bode well for quality control, at any rate. This is more noticable when
looking at output without sort/uniq being performed
The 1999 egcs result is most amusing :) Also in the output is a banner for
their code, the copyright on the banner is interesting:
MMM MMM KKK TTTTTTTTTTT KKK
MMMM MMMM KKK TTTTTTTTTTT KKK
MMM MMMM MMM III KKK KKK RRRRRR OOOOOO TTT III KKK KKK
MMM MM MMM III KKKKK RRR RRR OOO OOO TTT III KKKKK
MMM MMM III KKK KKK RRRRRR OOO OOO TTT III KKK KKK
MMM MMM III KKK KKK RRR RRR OOOOOO TTT III KKK KKK
MikroTik routerOS V2.4 (c) 1999-2001 http://mikrotik.com/
...
Tip: Read the manual.
Interesting kernel modules
In the strings output was a bunch of filenames, which look interesting, and
for further analysis. Output as follows:
ipt_helper.o
ipt_connbytes.o
ipt_length.o
ipt_physdev.o
ipt_ulog.o
ipt_snif.o
ipt_PASSTHR.o
ipt_p2p.o
addrlist.o
ipt_prmark.o
ipt_PRMARK.o
ipt_hsmark.o
logfw.o
trafflow.o
ulog.o
snif.o
logring.o
panics.o`
p2p.o
btest.o
checkmod.o
The addrlist.o, snif.o, logring.o, panics.o, p2p.o, btest.o,
checkmod.o don't look familar to me, based on my (somewhat limited)
experience with Linux Kernel. We'll look at this further later on and see
what's happening.
Interesting file names
Additionally in the strings output, there are some interesting filenames which
would be worth following up later on :)
telnet.jpg
winbox.jpg
roteros.dll
00roteros.info
winbox.exe
system.dll
system.info
Exploring the filesystem image
Another thanks to Steven for pointing out how
easy it is to mount partitions from disk images using the loopback device.
To get started, the partition table needs to be extracted, so we can calcuate
the applicable offset into the disk image that was created when the MikroTik
router package was installed in QEMU.
[box] $ /sbin/fdisk -l -u mikrotik.img
last_lba(): I don't know how to handle files with mode 81a4
You must set cylinders.
You can do this from the extra functions menu.
Disk mikrotik.img: 0 MB, 0 bytes
255 heads, 63 sectors/track, 0 cylinders, total 0 sectors
Units = sectors of 1 * 512 = 512 bytes
Device Boot Start End Blocks Id System
mikrotik.img1 * 1 1044224 522112 83 Linux
So according the above url, we can access the partition via the calcuation
start_sector * 512, which is just 512 :) Testing this out, we get:
[box] # mkdir /mnt/mikro
[box] # mount -o loop,offset=512 mikrotik.img /mnt/mikro/
[box] # dmesg | tail -n 6
loop: module loaded
kjournald starting. Commit interval 5 seconds
EXT3 FS on loop0, internal journal
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
SELinux: initialized (dev loop0, type ext3), uses xattr
Which looks good, so far.
The files that we can see are as follows:
# find .
.
./lost+found
./etc
./etc/lilo.conf
./etc/fstab
./etc/rc.d
./etc/rc.d/run.d
./etc/rc.d/run.d/S02panic
./etc/rc.d/run.d/C90restore
./etc/rc.d/run.d/S08softdog
./etc/rc.d/run.d/S06modules
./etc/rc.d/run.d/S06vlanmodules
./etc/rc.d/run.d/R99wblk
./etc/rc.d/run.d/S02logring
./etc/rc.d/run.d/S10nova
./etc/rc.d/run.d/K90nova
./etc/rc.d/run.d/S01init
./etc/rc.d/run.d/C20nova
./etc/rc.d/run.d/S05syslog
./etc/rc.d/run.d/S06ticker
./etc/rc.d/run.d/C95panic
./etc/rc.d/run.d/S01modules
./etc/rc.d/run.d/C99hwclock
./etc/rc.d/run.d/S09usb
./etc/rc.d/run.d/S09pcmcia
./etc/rc.d/run.d/S01hwclock
./etc/rc.d/run.d/S09sshd
./etc/rc.d/run.d/C30prism
./etc/rc.d/run.d/S06prismmodules
./etc/rc.d/rc.postinstall
./etc/rc.d/rc.start
./etc/rc.d/rc.install
./etc/rc.d/rc.sysinit
./etc/rc.d/rc.stop
./etc/profile
./etc/hosts
./etc/host.conf
./etc/passwd
./etc/license
./etc/ld.so.conf
./etc/services
./etc/protocols
./etc/nsswitch.conf
./etc/group
./etc/modules.conf
./etc/pcmcia
./etc/pcmcia/config
./etc/pcmcia/network
./etc/pcmcia/wavelan.conf
./etc/pcmcia/prism.conf
./etc/ld.so.cache
./etc/issue.net
./etc/ident
./etc/issue
./etc/ssh_host_key
./etc/ssh_host_key.pub
./etc/ssh_host_dsa_key
./etc/ssh_host_dsa_key.pub
./.nofsck
./proc
./nova
./nova/etc
./nova/etc/serial
./nova/etc/font_8x10.raw
./nova/etc/font_8x16.raw
./nova/etc/font_8x8.raw
./nova/etc/lognames
./nova/etc/upnp
./nova/etc/upnp/logo16.gif
./nova/etc/upnp/logo32.gif
./nova/etc/upnp/logo48.gif
./nova/etc/upnp/osinfo.xml
./nova/etc/upnp/wancommonifcfg.xml
./nova/etc/upnp/wanipconn.xml
./nova/etc/loader
./nova/etc/loader/system.x3
./nova/etc/loader/adv-tools.x3
./nova/etc/loader/calea.x3
./nova/etc/loader/dhcp.x3
./nova/etc/loader/security.x3
./nova/etc/loader/wireless.x3
./nova/etc/starter
./nova/etc/starter/system.x3
./nova/etc/starter/adv-tools.x3
./nova/etc/starter/calea.x3
./nova/etc/starter/dhcp.x3
./nova/etc/starter/security.x3
./nova/etc/modules
./nova/etc/modules/system.x3
./nova/etc/modules/wireless.x3
./nova/etc/pciinfo
./nova/etc/pciinfo/system.x3
./nova/etc/pciinfo/wireless.x3
./nova/etc/ports
./nova/etc/ports/system.x3
./nova/etc/services
./nova/etc/services/system.x3
./nova/etc/services/security.x3
./nova/etc/system_names
./nova/etc/system_names/system.x3
./nova/etc/user
./nova/etc/user/system.x3
./nova/etc/www
./nova/etc/www/system.x3
./nova/etc/net-remote
./nova/etc/net-remote/system.x3
./nova/etc/net-remote/wireless.x3
./nova/etc/log-prefix
./nova/etc/log-prefix/system.x3
./nova/etc/radius
./nova/etc/radius/system.x3
./nova/etc/logo.orig
./nova/etc/url.orig
./nova/etc/url
./nova/etc/logo
./nova/bin
./nova/bin/chfont
./nova/bin/milo
./nova/bin/loader
./nova/bin/starter
./nova/bin/stopper
./nova/bin/ping
./nova/bin/resolver
./nova/bin/moduler
./nova/bin/pcipnp
./nova/bin/hotplug
./nova/bin/havecardbus
./nova/bin/installer
./nova/bin/mkissue
./nova/bin/pckgchecker
./nova/bin/takeover
./nova/bin/eep
./nova/bin/convertbr
./nova/bin/convertqueue
./nova/bin/user
./nova/bin/log
./nova/bin/net
./nova/bin/blink
./nova/bin/netpcmcia
./nova/bin/undo
./nova/bin/btest2
./nova/bin/bserv2
./nova/bin/run_test
./nova/bin/traceroute
./nova/bin/bridge
./nova/bin/sys2
./nova/bin/updwblk
./nova/bin/ftpd
./nova/bin/watty
./nova/bin/email
./nova/bin/trafflow
./nova/bin/traflog
./nova/bin/traf_con
./nova/bin/watchdog
./nova/bin/portman
./nova/bin/sendmsg
./nova/bin/backup
./nova/bin/restore
./nova/bin/sermgr
./nova/bin/cerm
./nova/bin/licupgr
./nova/bin/telser
./nova/bin/pacman
./nova/bin/fileman
./nova/bin/traffic
./nova/bin/ippool
./nova/bin/keyman
./nova/bin/keyinfo
./nova/bin/arpd
./nova/bin/radius
./nova/bin/snmp
./nova/bin/mepty
./nova/bin/vrrp
./nova/bin/www
./nova/bin/upnp
./nova/bin/sniffer
./nova/bin/console
./nova/bin/login
./nova/bin/autoupdate
./nova/bin/mrgcx
./nova/bin/append
./nova/bin/info
./nova/bin/logmaker
./nova/bin/graphing
./nova/bin/discover
./nova/bin/mactel
./nova/bin/macping
./nova/bin/route
./nova/bin/mproxy
./nova/bin/wproxy
./nova/bin/webcfg
./nova/bin/socks
./nova/bin/panicsl
./nova/bin/ninstall
./nova/bin/memtest
./nova/bin/cputest
./nova/bin/sigwatch
./nova/bin/netwatch
./nova/bin/fping
./nova/bin/pspeed
./nova/bin/ddns
./nova/bin/macscan
./nova/bin/scanner
./nova/bin/calea
./nova/bin/dhcp
./nova/bin/dhcpclient
./nova/bin/sshd
./nova/bin/scp
./nova/bin/sftp-server
./nova/bin/ssh
./nova/bin/ssh-keygen
./nova/bin/sshell
./nova/bin/racoon
./nova/bin/wireless
./nova/bin/info2
./nova/bin/convertprofile
./nova/bin/convertprism
./nova/bin/convertacl
./nova/lib
./nova/lib/install
./nova/lib/install/profile.sh
./nova/lib/reset
./nova/lib/reset/all.reset
./nova/lib/reset/keep-users.reset
./nova/lib/www
./nova/lib/www/traflog.p
./nova/lib/www/index.p
./nova/lib/www/winbox.p
./nova/lib/www/webgraph.p
./nova/lib/www/webcfg.p
./nova/lib/snmp
./nova/lib/snmp/system.so
./nova/lib/snmp/ip.so
./nova/lib/snmp/interface.so
./nova/lib/snmp/bridge.so
./nova/lib/snmp/dhcp.so
./nova/lib/snmp/wireless.so
./nova/lib/console
./nova/lib/console/system.xi
./nova/lib/console/port-numbers.enum
./nova/lib/console/mac-protocol.enum
./nova/lib/console/ip-protocol.enum
./nova/lib/console/0.defaults.hlp
./nova/lib/console/bonding.hlp
./nova/lib/console/radius.hlp
./nova/lib/console/script.hlp
./nova/lib/console/system.hlp
./nova/lib/console/logo.txt.orig
./nova/lib/console/sublogo.txt.orig
./nova/lib/console/logo.txt.org
./nova/lib/console/advanced-tools.xi
./nova/lib/console/advanced-tools.hlp
./nova/lib/console/calea.xi
./nova/lib/console/calea.hlp
./nova/lib/console/dhcp.xi
./nova/lib/console/dhcp.hlp
./nova/lib/console/security.xi
./nova/lib/console/ipsec.hlp
./nova/lib/console/wireless.xi
./nova/lib/console/wavelan.hlp
./nova/lib/console/wireless.hlp
./nova/lib/console/logo.txt
./nova/lib/console/console.x3
./nova/lib/logmaker
./nova/lib/logmaker/0010..proc.lom
./nova/lib/logmaker/0020..acct.lom
./nova/lib/logmaker/0030..startup.lom
./nova/lib/logmaker/0040..debug.lom
./nova/lib/logmaker/0100.backup.lom
./nova/lib/logmaker/0100.resource.lom
./nova/lib/logmaker/0110.log.lom
./nova/lib/logmaker/1000.export.lom
./nova/lib/logmaker/2010.interface.lom
./nova/lib/logmaker/2020.address.lom
./nova/lib/logmaker/2025.arp.lom
./nova/lib/logmaker/2030.route.lom
./nova/lib/logmaker/2040.firewall.lom
./nova/lib/logmaker/2041.firewall-stats.lom
./nova/lib/logmaker/2045.bridge.lom
./nova/lib/logmaker/2050.queue.lom
./nova/lib/logmaker/2051.queue-packets.lom
./nova/lib/logmaker/2052.queue-bytes.lom
./nova/lib/logmaker/2060.ippool.lom
./nova/lib/logmaker/2061.certificate.lom
./nova/lib/logmaker/2070.neighbor.lom
./nova/lib/logmaker/2080.license.lom
./nova/lib/logmaker/2090.package.lom
./nova/lib/logmaker/2095.instchk.lom
./nova/lib/logmaker/2100.oops.lom
./nova/lib/logmaker/2105.backtrace.lom
./nova/lib/logmaker/2070.dhcp.lom
./nova/lib/logmaker/3010.wireless.lom
./nova/lib/logmaker/3011.wirelessdump.lom
./nova/lib/xmlnames2
./nova/lib/net
./nova/lib/net/aironet.np
./nova/lib/net/wvlan.np
./nova/logs
./nova/logs/VERSION
./nova/logs/backtrace.log
./nova/logs/temp_panic.log
./nova/logs/logsocket
./nova/store
./nova/store/net
./nova/store/net/simplequeues.idx
./nova/store/net/simplequeues.dat
./nova/store/net/simplequeues.lock
./nova/store/net/address-list.lock
./nova/store/net/address-list.dat
./nova/store/net/address-list.idx
./nova/store/net/queuetypes.lock
./nova/store/net/queuetypes.dat
./nova/store/net/queuetypes.idx
./nova/store/net/devices.lock
./nova/store/net/devices.dat
./nova/store/net/devices.idx
./nova/store/net/addrs.lock
./nova/store/net/addrs.dat
./nova/store/net/addrs.idx
./nova/store/net/module.lock
./nova/store/net/module.dat
./nova/store/net/module.idx
./nova/store/net/bridgefw.lock
./nova/store/net/bridgefw.dat
./nova/store/net/bridgefw.idx
./nova/store/net/ebt-filter.lock
./nova/store/net/ebt-filter.dat
./nova/store/net/ebt-filter.idx
./nova/store/net/ebt-calea.lock
./nova/store/net/ebt-calea.dat
./nova/store/net/ebt-calea.idx
./nova/store/net/ebt-nat.lock
./nova/store/net/ebt-nat.dat
./nova/store/net/ebt-nat.idx
./nova/store/net/ebt-broute.lock
./nova/store/net/ebt-broute.dat
./nova/store/net/ebt-broute.idx
./nova/store/net/raw.lock
./nova/store/net/raw.dat
./nova/store/net/raw.idx
./nova/store/net/mangle.lock
./nova/store/net/mangle.dat
./nova/store/net/mangle.idx
./nova/store/net/ipt-mangle.lock
./nova/store/net/ipt-mangle.dat
./nova/store/net/ipt-mangle.idx
./nova/store/net/ipt-calea.lock
./nova/store/net/ipt-calea.dat
./nova/store/net/ipt-calea.idx
./nova/store/net/ipt-filter.lock
./nova/store/net/ipt-filter.dat
./nova/store/net/ipt-filter.idx
./nova/store/net/ipt-nat.lock
./nova/store/net/ipt-nat.dat
./nova/store/net/ipt-nat.idx
./nova/store/net/arp.lock
./nova/store/net/arp.dat
./nova/store/net/arp.idx
./nova/store/net/queuetree.lock
./nova/store/net/queuetree.dat
./nova/store/net/queuetree.idx
./nova/store/net/routes.lock
./nova/store/net/routes.dat
./nova/store/net/routes.idx
./nova/store/mproxy.lock
./nova/store/mproxy.dat
./nova/store/mproxy.idx
./nova/store/system.lock
./nova/store/system.dat
./nova/store/system.idx
./nova/store/taffic-scripts.lock
./nova/store/taffic-scripts.dat
./nova/store/taffic-scripts.idx
./nova/store/watchdog.lock
./nova/store/watchdog.dat
./nova/store/watchdog.idx
./nova/store/chfont.lock
./nova/store/chfont.dat
./nova/store/chfont.idx
./nova/store/sermgr.lock
./nova/store/sermgr.dat
./nova/store/sermgr.idx
./nova/store/bserv.lock
./nova/store/bserv.dat
./nova/store/bserv.idx
./nova/store/macping.lock
./nova/store/macping.dat
./nova/store/macping.idx
./nova/store/bridgeports.lock
./nova/store/bridgeports.dat
./nova/store/bridgeports.idx
./nova/store/mactel.lock
./nova/store/mactel.dat
./nova/store/mactel.idx
./nova/store/discover.lock
./nova/store/discover.dat
./nova/store/discover.idx
./nova/store/ppacker.lock
./nova/store/ppacker.dat
./nova/store/ppacker.idx
./nova/store/log-actions.lock
./nova/store/log-actions.dat
./nova/store/log-actions.idx
./nova/store/log-rules.lock
./nova/store/log-rules.dat
./nova/store/log-rules.idx
./nova/store/serial-login.lock
./nova/store/serial-login.dat
./nova/store/serial-login.idx
./nova/store/sercon.lock
./nova/store/sercon.dat
./nova/store/sercon.idx
./nova/store/wirelessalign.lock
./nova/store/wirelessalign.dat
./nova/store/wirelessalign.idx
./nova/store/wireless.lock
./nova/store/wireless.dat
./nova/store/wireless.idx
./nova/store/wirelessccl.lock
./nova/store/wirelessccl.dat
./nova/store/wirelessccl.idx
./nova/store/wirelessprofile.lock
./nova/store/wirelessprofile.dat
./nova/store/wirelessprofile.idx
./nova/store/port_lock.lock
./nova/store/port_lock.dat
./nova/store/port_lock.idx
./nova/store/ssh-keys.lock
./nova/store/ssh-keys.dat
./nova/store/ssh-keys.idx
./nova/store/routing
./nova/store/routing/rule.lock
./nova/store/routing/rule.dat
./nova/store/routing/rule.idx
./nova/store/routing/filter.lock
./nova/store/routing/filter.dat
./nova/store/routing/filter.idx
./nova/store/wirelesssniffer.lock
./nova/store/wirelesssniffer.dat
./nova/store/wirelesssniffer.idx
./nova/store/wirelesssnoop.lock
./nova/store/wirelesssnoop.dat
./nova/store/wirelesssnoop.idx
./nova/store/group.lock
./nova/store/group.dat
./nova/store/group.idx
./nova/store/user.lock
./nova/store/user.dat
./nova/store/user.idx
./nova/store/user
./nova/store/user/aaa.lock
./nova/store/user/aaa.dat
./nova/store/user/aaa.idx
./nova/store/user/prefs
./nova/store/command
./nova/store/command/speclogin.lock
./nova/store/command/speclogin.dat
./nova/store/command/speclogin.idx
./nova/store/command/sysnote.lock
./nova/store/command/sysnote.dat
./nova/store/command/sysnote.idx
./nova/store/scripts.lock
./nova/store/scripts.dat
./nova/store/scripts.idx
./nova/store/scheduler.lock
./nova/store/scheduler.dat
./nova/store/scheduler.idx
./nova/store/echosave
./nova/run
./var
./var/pdb
./var/pdb/system
./var/pdb/system/oninstall
./var/pdb/system/onuninstall
./var/pdb/system/crc
./var/pdb/system/version
./var/pdb/system/deps
./var/pdb/system/files
./var/pdb/advanced-tools
./var/pdb/advanced-tools/oninstall
./var/pdb/advanced-tools/onuninstall
./var/pdb/advanced-tools/crc
./var/pdb/advanced-tools/version
./var/pdb/advanced-tools/deps
./var/pdb/advanced-tools/files
./var/pdb/calea
./var/pdb/calea/oninstall
./var/pdb/calea/onuninstall
./var/pdb/calea/crc
./var/pdb/calea/version
./var/pdb/calea/deps
./var/pdb/calea/files
./var/pdb/dhcp
./var/pdb/dhcp/oninstall
./var/pdb/dhcp/onuninstall
./var/pdb/dhcp/crc
./var/pdb/dhcp/version
./var/pdb/dhcp/deps
./var/pdb/dhcp/files
./var/pdb/security
./var/pdb/security/oninstall
./var/pdb/security/onuninstall
./var/pdb/security/crc
./var/pdb/security/version
./var/pdb/security/deps
./var/pdb/security/files
./var/pdb/wireless
./var/pdb/wireless/oninstall
./var/pdb/wireless/onuninstall
./var/pdb/wireless/crc
./var/pdb/wireless/version
./var/pdb/wireless/deps
./var/pdb/wireless/files
./var/deinstall
./var/lock
./var/pckg
./var/post
./var/run
./var/run/utmp
./var/tmp
./bin
./bin/bash_login
./bin/mlogin
./bin/ask
./bin/catlog
./bin/pakp
./bin/pacd
./bin/busybox
./bin/basename
./bin/cat
./bin/chmod
./bin/chown
./bin/cp
./bin/date
./bin/expr
./bin/find
./bin/hostname
./bin/ln
./bin/mkdir
./bin/mknod
./bin/mv
./bin/rm
./bin/touch
./bin/uname
./bin/usleep
./bin/mount
./bin/umount
./bin/echo
./bin/[
./bin/test
./bin/ash
./bin/bash
./bin/telnet
./bin/gosh
./bin/login
./bin/shell
./bin/burnP5
./bin/burnP6
./bin/burnK6
./bin/burnK7
./home
./home/web
./home/web/webpda
./home/web/webpda/ip_address.html
./home/web/webpda/simple_queues.html
./home/web/webpda/reset.html
./home/web/webpda/status.html
./home/web/webpda/style.css
./home/web/webpda/snooper.html
./home/web/webpda/common.html
./home/web/webpda/bridge.html
./home/web/webpda/buttons.html
./home/web/webpda/edit_lease.html
./home/web/webpda/ppp.html
./home/web/webpda/interface.html
./home/web/webpda/top.html
./home/web/webpda/password.html
./home/web/webpda/edit_simple_queue.html
./home/web/webpda/bground.jpg
./home/web/webpda/routes.html
./home/web/webpda/registration_table.html
./home/web/webpda/edit_ip_address.html
./home/web/webpda/message.html
./home/web/webpda/upgrade.html
./home/web/webpda/dashboard.html
./home/web/webpda/access_list.html
./home/web/webpda/wireless.html
./home/web/webpda/toplogo.jpg
./home/web/webpda/edit_access_list.html
./home/web/webpda/edit_route.html
./home/web/webpda/dhcp_server.html
./home/web/webpda/firewall.html
./home/web/webpda/edit_interface_name.html
./home/web/webpda/login.html
./home/web/webpda/fail.html
./home/web/help
./home/web/help/openssh.html
./home/web/help/ldso-license.html
./home/web/help/expat.html
./home/web/help/racoon.html
./home/web/help/license.html
./home/web/help/gpl.html
./home/web/help/openssl.html
./home/web/help/mpl.html
./home/web/help/telnet.html
./home/web/help/pcmcia.html
./home/web/help/lgpl.html
./home/web/help/ppp.html
./home/web/help/ntp.html
./home/web/help/ldso.html
./home/web/help/stl.html
./home/web/help/bsd.html
./home/web/img
./home/web/img/yellow.gif
./home/web/img/mem.gif
./home/web/img/client.gif
./home/web/img/station.gif
./home/web/img/fishy.gif
./home/web/img/edit.gif
./home/web/img/enable.gif
./home/web/img/static.gif
./home/web/img/cpu.gif
./home/web/img/rx.gif
./home/web/img/tx.gif
./home/web/img/clock.gif
./home/web/img/remove.gif
./home/web/img/toplogo.jpg
./home/web/img/dhcp1.gif
./home/web/img/hdd.gif
./home/web/img/ap.gif
./home/web/img/dhcp2.gif
./home/web/img/green.gif
./home/web/img/gray.gif
./home/web/img/connected.gif
./home/web/img/disable.gif
./home/web/img/uptime.gif
./home/web/img/bar.gif
./home/web/img/calendar.gif
./home/web/img/red.gif
./home/web/webcfg
./home/web/webcfg/simple_queues.html
./home/web/webcfg/reset.html
./home/web/webcfg/status.html
./home/web/webcfg/pppoe.html
./home/web/webcfg/style.css
./home/web/webcfg/snooper.html
./home/web/webcfg/jsrsClient.js
./home/web/webcfg/common.html
./home/web/webcfg/reboot.html
./home/web/webcfg/edit_lease.html
./home/web/webcfg/buttons.html
./home/web/webcfg/ppp.html
./home/web/webcfg/common.js
./home/web/webcfg/interface.html
./home/web/webcfg/top.html
./home/web/webcfg/password.html
./home/web/webcfg/edit_simple_queue.html
./home/web/webcfg/bground.jpg
./home/web/webcfg/registration_table.html
./home/web/webcfg/routes.html
./home/web/webcfg/edit_ip_address.html
./home/web/webcfg/message.html
./home/web/webcfg/upgrade.html
./home/web/webcfg/dashboard.html
./home/web/webcfg/access_list.html
./home/web/webcfg/wireless.html
./home/web/webcfg/snooper_frame.html
./home/web/webcfg/toplogo.jpg
./home/web/webcfg/edit_access_list.html
./home/web/webcfg/snooper_refresh.html
./home/web/webcfg/edit_route.html
./home/web/webcfg/dhcp_server.html
./home/web/webcfg/firewall.html
./home/web/webcfg/edit_interface_name.html
./home/web/webcfg/login.html
./home/web/webcfg/fail.html
./home/web/toplogo.jpg
./home/web/mtlogos.jpg
./home/web/webboxs.jpg
./home/web/grap.jpg
./home/web/winboxs.jpg
./home/web/docss.jpg
./home/web/telnets.jpg
./home/web/index2.html.orig
./home/web/winbox
./home/web/winbox/roteros.dll
./home/web/winbox/00roteros.info
./home/web/winbox/winbox.exe
./home/web/winbox/system.dll
./home/web/winbox/system.info
./home/web/winbox/index
./home/web/winbox/advtool.dll
./home/web/winbox/advtool.info
./home/web/winbox/dhcp.dll
./home/web/winbox/dhcp.info
./home/web/winbox/secure.dll
./home/web/winbox/secure.info
./home/web/winbox/wlan2.dll
./home/web/winbox/wlan2.info
./home/web/logo.gif
./home/web/telnet.jpg
./home/web/winbox.jpg
./home/web/index2.html
./sbin
./sbin/halt
./sbin/reboot
./sbin/ldconfig
./sbin/hwclock
./sbin/cardmgr
./sbin/fsck.ext2
./sbin/fsck
./sbin/tune2fs
./sbin/diskman
./sbin/init
./sbin/loconf
./sbin/shif
./sbin/syslogd
./sbin/klogd
./sbin/chroot
./sbin/modprobe
./sbin/insmod
./sbin/rmmod
./sbin/lsmod
./sbin/clock
./sbin/fsck.ext3
./sbin/hotplug
./boot
./boot/milo.conf
./boot/vmlinuz
./boot/initrd.rgz
./boot/map
./boot/memtest.bin
./lib
./lib/libnss_files-2.3.6.so
./lib/ld-2.3.6.so
./lib/libgcc_s.so.1
./lib/libresolv-2.3.6.so
./lib/libm-2.3.6.so
./lib/libnsl-2.3.6.so
./lib/libutil-2.3.6.so
./lib/libpthread-0.10.so
./lib/libdl-2.3.6.so
./lib/libcrypt-2.3.6.so
./lib/libc-2.3.6.so
./lib/modules
./lib/modules/2.4.31
./lib/modules/2.4.31/drivers
./lib/modules/2.4.31/drivers/net
./lib/modules/2.4.31/drivers/net/mii.o
./lib/modules/2.4.31/drivers/net/slhc.o
./lib/modules/2.4.31/drivers/net/via-rhine.o
./lib/modules/2.4.31/drivers/net/hp100.o
./lib/modules/2.4.31/drivers/net/8390.o
./lib/modules/2.4.31/drivers/net/ne2k-pci.o
./lib/modules/2.4.31/drivers/net/ne.o
./lib/modules/2.4.31/drivers/net/lance.o
./lib/modules/2.4.31/drivers/net/3c509.o
./lib/modules/2.4.31/drivers/net/3c59x.o
./lib/modules/2.4.31/drivers/net/sis900.o
./lib/modules/2.4.31/drivers/net/8139too.o
./lib/modules/2.4.31/drivers/net/pcnet32.o
./lib/modules/2.4.31/drivers/net/winbond-840.o
./lib/modules/2.4.31/drivers/net/tulip
./lib/modules/2.4.31/drivers/net/tulip/tulip.o
./lib/modules/2.4.31/drivers/net/ns83820.o
./lib/modules/2.4.31/drivers/net/sundance.o
./lib/modules/2.4.31/drivers/net/dmfe.o
./lib/modules/2.4.31/drivers/net/e100
./lib/modules/2.4.31/drivers/net/e100/e100.o
./lib/modules/2.4.31/drivers/net/tg3.o
./lib/modules/2.4.31/drivers/net/natsemi.o
./lib/modules/2.4.31/drivers/net/b44.o
./lib/modules/2.4.31/drivers/net/tlan.o
./lib/modules/2.4.31/drivers/net/bonding
./lib/modules/2.4.31/drivers/net/bonding/bonding.o
./lib/modules/2.4.31/drivers/net/imq.o
./lib/modules/2.4.31/drivers/net/pcmcia
./lib/modules/2.4.31/drivers/net/pcmcia/pcnet_cs.o
./lib/modules/2.4.31/drivers/net/il.o
./lib/modules/2.4.31/drivers/net/sk98lin.o
./lib/modules/2.4.31/drivers/net/velocityget.o
./lib/modules/2.4.31/drivers/net/r8169.o
./lib/modules/2.4.31/drivers/net/e1000.o
./lib/modules/2.4.31/drivers/net/ipsec.o
./lib/modules/2.4.31/drivers/net/wireless
./lib/modules/2.4.31/drivers/net/wireless/hermes.o
./lib/modules/2.4.31/drivers/net/wireless/orinoco.o
./lib/modules/2.4.31/drivers/net/wireless/orinoco_cs.o
./lib/modules/2.4.31/drivers/char
./lib/modules/2.4.31/drivers/char/softdog.o
./lib/modules/2.4.31/drivers/char/cyclades.o
./lib/modules/2.4.31/drivers/char/pcmcia
./lib/modules/2.4.31/drivers/char/pcmcia/serial_cs.o
./lib/modules/2.4.31/drivers/char/ticker.o
./lib/modules/2.4.31/drivers/char/databooster.o
./lib/modules/2.4.31/drivers/char/mxser.o
./lib/modules/2.4.31/drivers/pnp
./lib/modules/2.4.31/drivers/pnp/isa-pnp.o
./lib/modules/2.4.31/drivers/usb
./lib/modules/2.4.31/drivers/usb/acm.o
./lib/modules/2.4.31/drivers/usb/usbcore.o
./lib/modules/2.4.31/drivers/usb/host
./lib/modules/2.4.31/drivers/usb/host/usb-uhci.o
./lib/modules/2.4.31/drivers/usb/host/usb-ohci.o
./lib/modules/2.4.31/drivers/usb/hid.o
./lib/modules/2.4.31/drivers/usb/rtl8150.o
./lib/modules/2.4.31/drivers/usb/pegasus.o
./lib/modules/2.4.31/drivers/usb/serial
./lib/modules/2.4.31/drivers/usb/serial/usbserial.o
./lib/modules/2.4.31/drivers/usb/serial/mtusbserial.o
./lib/modules/2.4.31/drivers/pcmcia
./lib/modules/2.4.31/drivers/pcmcia/yenta_socket.o
./lib/modules/2.4.31/drivers/pcmcia/ds.o
./lib/modules/2.4.31/drivers/pcmcia/pcmcia_core.o
./lib/modules/2.4.31/drivers/pcmcia/i82365.o
./lib/modules/2.4.31/lib
./lib/modules/2.4.31/lib/crc32.o
./lib/modules/2.4.31/net
./lib/modules/2.4.31/net/8021q
./lib/modules/2.4.31/net/8021q/8021q.o
./lib/modules/2.4.31/net/bridge
./lib/modules/2.4.31/net/bridge/bridge.o
./lib/modules/2.4.31/net/bridge/netfilter
./lib/modules/2.4.31/net/bridge/netfilter/ebtables.o
./lib/modules/2.4.31/net/bridge/netfilter/ebtable_broute.o
./lib/modules/2.4.31/net/bridge/netfilter/ebtable_filter.o
./lib/modules/2.4.31/net/bridge/netfilter/ebtable_nat.o
./lib/modules/2.4.31/net/bridge/netfilter/ebt_802_3.o
./lib/modules/2.4.31/net/bridge/netfilter/ebt_arp.o
./lib/modules/2.4.31/net/bridge/netfilter/ebt_arpreply.o
./lib/modules/2.4.31/net/bridge/netfilter/ebt_dnat.o
./lib/modules/2.4.31/net/bridge/netfilter/ebt_ip.o
./lib/modules/2.4.31/net/bridge/netfilter/ebt_limit.o
./lib/modules/2.4.31/net/bridge/netfilter/ebt_mark.o
./lib/modules/2.4.31/net/bridge/netfilter/ebt_mark_m.o
./lib/modules/2.4.31/net/bridge/netfilter/ebt_pkttype.o
./lib/modules/2.4.31/net/bridge/netfilter/ebt_redirect.o
./lib/modules/2.4.31/net/bridge/netfilter/ebt_snat.o
./lib/modules/2.4.31/net/bridge/netfilter/ebt_stp.o
./lib/modules/2.4.31/net/bridge/netfilter/ebt_vlan.o
./lib/modules/2.4.31/net/bridge/ebt_ulog.o
./lib/modules/2.4.31/net/bridge/ebt_snif.o
./lib/modules/2.4.31/net/unix
./lib/modules/2.4.31/net/unix/unix.o
./lib/modules/2.4.31/net/packet
./lib/modules/2.4.31/net/packet/af_packet.o
./lib/modules/2.4.31/net/sched
./lib/modules/2.4.31/net/sched/sch_htb.o
./lib/modules/2.4.31/net/sched/sch_red.o
./lib/modules/2.4.31/net/sched/sch_sfq.o
./lib/modules/2.4.31/net/sched/cls_fw.o
./lib/modules/2.4.31/net/sched/cls_linear.o
./lib/modules/2.4.31/net/sched/sch_agr.o
./lib/modules/2.4.31/net/sched/proto_agr.o
./lib/modules/2.4.31/net/sched/sch_pcq.o
./lib/modules/2.4.31/net/sched/sch_rate.o
./lib/modules/2.4.31/net/ipv4
./lib/modules/2.4.31/net/ipv4/ipip.o
./lib/modules/2.4.31/net/ipv4/netfilter
./lib/modules/2.4.31/net/ipv4/netfilter/ip_tables.o
./lib/modules/2.4.31/net/ipv4/netfilter/iptable_filter.o
./lib/modules/2.4.31/net/ipv4/netfilter/iptable_nat.o
./lib/modules/2.4.31/net/ipv4/netfilter/iptable_mangle.o
./lib/modules/2.4.31/net/ipv4/netfilter/iptable_raw.o
./lib/modules/2.4.31/net/ipv4/netfilter/nfnetlink.o
./lib/modules/2.4.31/net/ipv4/netfilter/ip_nat_ftp.o
./lib/modules/2.4.31/net/ipv4/netfilter/ip_nat_irc.o
./lib/modules/2.4.31/net/ipv4/netfilter/ip_nat_h323.o
./lib/modules/2.4.31/net/ipv4/netfilter/ip_nat_proto_gre.o
./lib/modules/2.4.31/net/ipv4/netfilter/ip_nat_pptp.o
./lib/modules/2.4.31/net/ipv4/netfilter/ip_nat_quake3.o
./lib/modules/2.4.31/net/ipv4/netfilter/ip_nat_tftp.o
./lib/modules/2.4.31/net/ipv4/netfilter/ip_conntrack.o
./lib/modules/2.4.31/net/ipv4/netfilter/ip_conntrack_ftp.o
./lib/modules/2.4.31/net/ipv4/netfilter/ip_conntrack_irc.o
./lib/modules/2.4.31/net/ipv4/netfilter/ip_conntrack_h323.o
./lib/modules/2.4.31/net/ipv4/netfilter/ip_conntrack_proto_gre.o
./lib/modules/2.4.31/net/ipv4/netfilter/ip_conntrack_pptp.o
./lib/modules/2.4.31/net/ipv4/netfilter/ip_conntrack_quake3.o
./lib/modules/2.4.31/net/ipv4/netfilter/ip_conntrack_tftp.o
./lib/modules/2.4.31/net/ipv4/netfilter/nfnetlink_conntrack.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_MASQUERADE.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_REJECT.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_TCPMSS.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_REDIRECT.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_MARK.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_FTOS.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_NOTRACK.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_CONNMARK.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_IPV4OPTSSTRIP.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_TARPIT.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_SAME.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_NETMAP.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_TTL.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_limit.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_mac.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_mark.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_tos.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_connlimit.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_state.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_connmark.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_string.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_iprange.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_dstlimit.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_time.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_random.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_psd.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_nth.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_ipv4options.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_addrtype.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_tcpmss.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_helper.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_connbytes.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_length.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_physdev.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_ulog.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_snif.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_PASSTHR.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_p2p.o
./lib/modules/2.4.31/net/ipv4/netfilter/addrlist.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_prmark.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_PRMARK.o
./lib/modules/2.4.31/net/ipv4/netfilter/ipt_hsmark.o
./lib/modules/2.4.31/net/ipv4/gre.o
./lib/modules/2.4.31/net/ipv4/eogre.o
./lib/modules/2.4.31/net/ath.o
./lib/modules/2.4.31/net/a5212_pci.o
./lib/modules/2.4.31/net/a5212.o
./lib/modules/2.4.31/net/a5211.o
./lib/modules/2.4.31/net/a5211_pci.o
./lib/modules/2.4.31/net/prism.o
./lib/modules/2.4.31/net/prism2_cs.o
./lib/modules/2.4.31/net/prism2_pci.o
./lib/modules/2.4.31/net/aironet.o
./lib/modules/2.4.31/net/anet-isa.o
./lib/modules/2.4.31/net/anet-pci.o
./lib/modules/2.4.31/net/a5210.o
./lib/modules/2.4.31/net/a5210_pci.o
./lib/modules/2.4.31/misc
./lib/modules/2.4.31/misc/logfw.o
./lib/modules/2.4.31/misc/trafflow.o
./lib/modules/2.4.31/misc/ulog.o
./lib/modules/2.4.31/misc/snif.o
./lib/modules/2.4.31/misc/logring.o
./lib/modules/2.4.31/misc/panics.o
./lib/modules/2.4.31/misc/p2p.o
./lib/modules/2.4.31/misc/btest.o
./lib/modules/2.4.31/misc/checkmod.o
./lib/modules/2.4.31/misc/wlan.o
./lib/modules/2.4.31/modules.dep.system
./lib/modules/2.4.31/modules.dep.security
./lib/modules/2.4.31/pcmcia
./lib/modules/2.4.31/pcmcia/arlan_cs.o
./lib/modules/2.4.31/modules.dep.wireless
./lib/modules/2.4.31/modules.dep
./lib/libuc++.so
./lib/libz.so
./lib/libumsg.so
./lib/libuxml++.so
./lib/libinstall.so
./lib/libufiber.so
./lib/libubox.so
./lib/liburadius.so
./lib/libnetext.so
./lib/libmac.so
./lib/libauth.so
./lib/libsnmp.so
./lib/libwebs.so
./lib/libwebc.so
./lib/libcrypto.so.0.9.8
./lib/libssl.so.0.9.8
./lib/libexpat.so.1.5.0
./lib/libexpat.so.1
./lib/libc.so.6
./lib/libcrypt.so.1
./lib/libdl.so.2
./lib/libpthread.so.0
./lib/libutil.so.1
./lib/libnsl.so.1
./lib/libm.so.6
./lib/libresolv.so.2
./lib/ld-linux.so.2
./lib/libnss_files.so.2
./lib/libudhcp.so
./lib/libssh.so
./lib/libauth2.so
./old
./old/system
./old/system/update
./old/system/halt
./old/system/reboot
./old/wireless
./old/wireless/update
./usr
./usr/sbin
./usr/sbin/telnetd
./usr/lib
./dev
./dev/autobaud
./dev/boot
./dev/btest
./dev/console
./dev/cua0
./dev/cua1
./dev/fancon
./dev/fd
./dev/fd0
./dev/flash
./dev/full
./dev/hda
./dev/hda1
./dev/hda2
./dev/hdb
./dev/hdb1
./dev/hdb2
./dev/hdc
./dev/hdc1
./dev/hdc2
./dev/hdd
./dev/hdd1
./dev/hdd2
./dev/hiddev0
./dev/hiddev1
./dev/hiddev10
./dev/hiddev11
./dev/hiddev12
./dev/hiddev13
./dev/hiddev14
./dev/hiddev15
./dev/hiddev2
./dev/hiddev3
./dev/hiddev4
./dev/hiddev5
./dev/hiddev6
./dev/hiddev7
./dev/hiddev8
./dev/hiddev9
./dev/kmem
./dev/log0
./dev/log1
./dev/log10
./dev/log100
./dev/log101
./dev/log102
./dev/log103
./dev/log104
./dev/log105
./dev/log106
./dev/log107
./dev/log108
./dev/log109
./dev/log11
./dev/log110
./dev/log111
./dev/log112
./dev/log113
./dev/log114
./dev/log115
./dev/log116
./dev/log117
./dev/log118
./dev/log119
./dev/log12
./dev/log120
./dev/log121
./dev/log2
./dev/log122
./dev/log123
./dev/log124
./dev/log125
./dev/log126
./dev/log127
./dev/log13
./dev/log14
./dev/log15
./dev/log16
./dev/log17
./dev/log18
./dev/log19
./dev/log20
./dev/log21
./dev/log22
./dev/log23
./dev/log24
./dev/log240
./dev/log241
./dev/log242
./dev/log243
./dev/log244
./dev/log245
./dev/log246
./dev/log247
./dev/log248
./dev/log249
./dev/log25
./dev/log255
./dev/log26
./dev/log27
./dev/log28
./dev/log29
./dev/log3
./dev/log30
./dev/log31
./dev/log32
./dev/log33
./dev/log34
./dev/log35
./dev/log36
./dev/log37
./dev/log38
./dev/log39
./dev/log4
./dev/log40
./dev/log41
./dev/log42
./dev/log43
./dev/log44
./dev/log45
./dev/log46
./dev/log47
./dev/log48
./dev/log49
./dev/log5
./dev/log50
./dev/log51
./dev/log52
./dev/log53
./dev/log54
./dev/log55
./dev/log56
./dev/log6
./dev/log57
./dev/log58
./dev/log59
./dev/log60
./dev/log61
./dev/log62
./dev/log63
./dev/log64
./dev/log65
./dev/log66
./dev/log67
./dev/log68
./dev/log69
./dev/log7
./dev/log70
./dev/log71
./dev/log72
./dev/log73
./dev/log74
./dev/log75
./dev/log76
./dev/log77
./dev/log78
./dev/log79
./dev/log8
./dev/log80
./dev/log81
./dev/log82
./dev/log83
./dev/log84
./dev/log85
./dev/log86
./dev/log87
./dev/log88
./dev/log89
./dev/log9
./dev/log90
./dev/log91
./dev/log92
./dev/log93
./dev/log94
./dev/log95
./dev/log96
./dev/log97
./dev/log98
./dev/log99
./dev/logfw
./dev/mem
./dev/null
./dev/panics
./dev/port
./dev/ptmx
./dev/pts
./dev/ram
./dev/ram1
./dev/random
./dev/root
./dev/rtc
./dev/ticker
./dev/trafflow
./dev/tty
./dev/tty0
./dev/tty1
./dev/tty2
./dev/tty3
./dev/tty4
./dev/tty5
./dev/tty6
./dev/tty7
./dev/tty8
./dev/tty9
./dev/ttyACM0
./dev/ttyACM1
./dev/ttyACM10
./dev/ttyACM11
./dev/ttyACM12
./dev/ttyACM13
./dev/ttyACM14
./dev/ttyACM15
./dev/ttyACM16
./dev/ttyACM17
./dev/ttyACM18
./dev/ttyACM19
./dev/ttyACM2
./dev/ttyACM20
./dev/ttyACM21
./dev/ttyACM22
./dev/ttyACM23
./dev/ttyACM24
./dev/ttyACM25
./dev/ttyACM26
./dev/ttyACM27
./dev/ttyACM28
./dev/ttyACM29
./dev/ttyACM3
./dev/ttyACM30
./dev/ttyACM31
./dev/ttyACM4
./dev/ttyACM5
./dev/ttyACM6
./dev/ttyACM7
./dev/ttyACM8
./dev/ttyACM9
./dev/ttyC0
./dev/ttyC1
./dev/ttyC10
./dev/ttyC11
./dev/ttyC12
./dev/ttyC13
./dev/ttyC14
./dev/ttyC15
./dev/ttyC16
./dev/ttyC17
./dev/ttyC18
./dev/ttyC19
./dev/ttyC2
./dev/ttyC20
./dev/ttyC21
./dev/ttyC22
./dev/ttyC23
./dev/ttyC24
./dev/ttyC25
./dev/ttyC26
./dev/ttyC27
./dev/ttyC28
./dev/ttyC29
./dev/ttyC3
./dev/ttyC30
./dev/ttyC31
./dev/ttyC4
./dev/ttyC5
./dev/ttyC6
./dev/ttyC7
./dev/ttyC8
./dev/ttyC9
./dev/ttyDB0
./dev/ttyDB1
./dev/ttyDB10
./dev/ttyDB11
./dev/ttyDB12
./dev/ttyDB13
./dev/ttyDB14
./dev/ttyDB15
./dev/ttyDB16
./dev/ttyDB17
./dev/ttyDB18
./dev/ttyDB19
./dev/ttyDB2
./dev/ttyDB20
./dev/ttyDB21
./dev/ttyDB22
./dev/ttyDB23
./dev/ttyDB24
./dev/ttyDB25
./dev/ttyDB26
./dev/ttyDB27
./dev/ttyDB28
./dev/ttyDB29
./dev/ttyDB3
./dev/ttyDB30
./dev/ttyDB31
./dev/ttyDB4
./dev/ttyDB5
./dev/ttyDB6
./dev/ttyDB7
./dev/ttyDB8
./dev/ttyDB9
./dev/ttyS0
./dev/ttyS1
./dev/ttyS16
./dev/ttyS17
./dev/ttyS18
./dev/ttyS19
./dev/ttyS2
./dev/ttyS3
./dev/ttySI0
./dev/ttySI1
./dev/ttySI10
./dev/ttySI11
./dev/ttySI12
./dev/ttySI13
./dev/ttySI14
./dev/ttySI15
./dev/ttySI16
./dev/ttySI17
./dev/ttySI18
./dev/ttySI19
./dev/ttySI2
./dev/ttySI20
./dev/ttySI21
./dev/ttySI22
./dev/ttySI23
./dev/ttySI24
./dev/ttySI25
./dev/ttySI26
./dev/ttySI27
./dev/ttySI28
./dev/ttySI29
./dev/ttySI3
./dev/ttySI30
./dev/ttySI31
./dev/ttySI4
./dev/ttySI5
./dev/ttySI6
./dev/ttySI7
./dev/ttySI8
./dev/ttySI9
./dev/ttyUSB0
./dev/ttyUSB1
./dev/ttyUSB10
./dev/ttyUSB11
./dev/ttyUSB12
./dev/ttyUSB13
./dev/ttyUSB14
./dev/ttyUSB15
./dev/ttyUSB2
./dev/ttyUSB3
./dev/ttyUSB4
./dev/ttyUSB5
./dev/ttyUSB6
./dev/ttyUSB7
./dev/ttyUSB8
./dev/ttyUSB9
./dev/ttyp0
./dev/ttyp1
./dev/ttyp2
./dev/ttyp3
./dev/ttyp4
./dev/ttyp5
./dev/ttyp6
./dev/ttyp7
./dev/ttyp8
./dev/ttyp9
./dev/ttypa
./dev/ttypb
./dev/ttypc
./dev/ttypd
./dev/ttype
./dev/ttypf
./dev/urandom
./dev/watchdog
./dev/zero
./dev/log
./initrd
./tmp
./tmp/pacdsock
./tmp/novasock
./tmp/.fibnet
./tmp/zserv.api
./root
./root/prism.hex
./CDINSTALL
./ram
./.asked
This has opened up the embedded device rather nicely for us, which will makes
things a lot easier for us.
Exploring the kernel modules
As mentioned above, a bunch of the kernel modules appear to have non-standard
filenames (which would appear to be custom modules). Let's have a more through
look at them (with additional formatting to make it easier to read):
[box] $ find . -type f -iname *o -exec modinfo '{}' \;
filename: ./drivers/net/mii.o
kernel_version: 2.4.31
author: Jeff Garzik <jgarzik@pobox.com>
description: MII hardware support library
license: GPL
filename: ./drivers/net/slhc.o
kernel_version: 2.4.31
license: Dual BSD/GPL
filename: ./drivers/net/via-rhine.o
kernel_version: 2.4.31
author: Donald Becker <becker@scyld.com>
description: VIA Rhine PCI Fast Ethernet driver
license: GPL
parm_max_interrupt_work:i
parm_debug: i
parm_disable_sleep_mode:i
parm_rx_copybreak:i
parm_backoff: i
parm_options: 1-8i
parm_full_duplex:1-8i
parm_desc_max_interrupt_work:VIA Rhine maximum events handled per interrupt
parm_desc_debug:VIA Rhine debug level (0-7)
parm_desc_disable_sleep_mode:VIA Rhine d3 state disable (1)
parm_desc_rx_copybreak:VIA Rhine copy breakpoint for copy-only-tiny-frames
parm_desc_backoff:VIA Rhine: Bits 0-3: backoff algorithm
parm_desc_options:VIA Rhine: Bits 0-3: media type, bit 17: full duplex
parm_desc_full_duplex:VIA Rhine full duplex setting(s) (1)
filename: ./drivers/net/hp100.o
kernel_version: 2.4.31
parm_hp100_rx_ratio:1i
parm_hp100_priority_tx:1i
parm_hp100_mode:1i
license: GPL
author: Jaroslav Kysela <perex@suse.cz>, Siegfried "Frieder" Loeffler (dg1sek) <floeff@mathematik.uni-stuttgart.de>
description: HP CASCADE Architecture Driver for 100VG-AnyLan Network Adapters
parm_hp100_port:1-5i
parm_hp100_name:1-5c16
filename: ./drivers/net/8390.o
kernel_version: 2.4.31
license: GPL
filename: ./drivers/net/ne2k-pci.o
kernel_version: 2.4.31
author: Donald Becker / Paul Gortmaker
description: PCI NE2000 clone driver
license: GPL
parm_debug: i
parm_options: 1-8i
parm_full_duplex:1-8i
parm_desc_debug:debug level (1-2)
parm_desc_options:Bit 5: full duplex
parm_desc_full_duplex:full duplex setting(s) (1)
filename: ./drivers/net/ne.o
kernel_version: 2.4.31
parm_io: 1-4i
parm_irq: 1-4i
parm_bad: 1-4i
parm_desc_io: I/O base address(es),required
parm_desc_irq: IRQ number(s)
parm_desc_bad: Accept card(s) with bad signatures
description: NE1000/NE2000 ISA/PnP Ethernet driver
license: GPL
filename: ./drivers/net/lance.o
kernel_version: 2.4.31
parm_io: 1-8i
parm_dma: 1-8i
parm_irq: 1-8i
parm_lance_debug:i
parm_desc_io: LANCE/PCnet I/O base address(es),required
parm_desc_dma: LANCE/PCnet ISA DMA channel (ignored for some devices)
parm_desc_irq: LANCE/PCnet IRQ number (ignored for some devices)
parm_desc_lance_debug:LANCE/PCnet debug level (0-7)
license: GPL
filename: ./drivers/net/3c509.o
kernel_version: 2.4.31
parm_debug: i
parm_irq: 1-8i
parm_xcvr: 1-12i
parm_max_interrupt_work:i
parm_desc_debug:debug level (0-6)
parm_desc_irq: IRQ number(s) (assigned)
parm_desc_xcvr: tranceiver(s) (0=internal, 1=external)
parm_desc_max_interrupt_work:maximum events handled per interrupt
description: 3Com Etherlink III (3c509, 3c509B) ISA/PnP ethernet driver
license: GPL
filename: ./drivers/net/3c59x.o
kernel_version: 2.4.31
author: Donald Becker <becker@scyld.com>
description: 3Com 3c59x/3c9xx ethernet driver LK1.1.18 1 July 2002
license: GPL
parm_debug: i
parm_global_options:i
parm_options: 1-8i
parm_global_full_duplex:i
parm_full_duplex:1-8i
parm_hw_checksums:1-8i
parm_flow_ctrl: 1-8i
parm_global_enable_wol:i
parm_enable_wol:1-8i
parm_rx_copybreak:i
parm_max_interrupt_work:i
parm_compaq_ioaddr:i
parm_compaq_irq:i
parm_compaq_device_id:i
parm_watchdog: i
parm_desc_debug:3c59x debug level (0-6)
parm_desc_options:3c59x: Bits 0-3: media type, bit 4: bus mastering, bit 9: full duplex
parm_desc_global_options:3c59x: same as options, but applies to all NICs if options is unset
parm_desc_full_duplex:3c59x full duplex setting(s) (1)
parm_desc_global_full_duplex:3c59x: same as full_duplex, but applies to all NICs if options is unset
parm_desc_hw_checksums:3c59x Hardware checksum checking by adapter(s) (0-1)
parm_desc_flow_ctrl:3c59x 802.3x flow control usage (PAUSE only) (0-1)
parm_desc_enable_wol:3c59x: Turn on Wake-on-LAN for adapter(s) (0-1)
parm_desc_global_enable_wol:3c59x: same as enable_wol, but applies to all NICs if options is unset
parm_desc_rx_copybreak:3c59x copy breakpoint for copy-only-tiny-frames
parm_desc_max_interrupt_work:3c59x maximum events handled per interrupt
p