Firmware Modification Kit
Giving YOU the power to customize your firmware
documentation version 0.51 beta
all scripts, untrx, wrt_vx_imgtool, and docs (c) 2006,2007 Jeremy Collake <jeremy.collake@gmail.com>
http://www.bitsum.com
other portions (c) their respective holders
Contents
I. Description
II. Why is this useful?
III. Where to get the kit
IV. Requirements
V. Compatibility
VI. Instructions
A.
extracting firmware
B.
modifying
firmware
1.
installing packages
2.
removing packages
C.
re-building firmware
VII. Caveats
A. binary
compatibility issues
B. maximum firmware size
C. solutions to file
system incompatibilities
VIII. Acknowledgements and more
IX. Links
X. Revision history
XI. About the author
I. Description
This kit gives the user the ability to make changes to a firmware image without
recompiling the firmware sources. It works by extracting the firmware into
its component parts, then extracting the file system image (assuming its in one
the supported formats). The user can then make modifications
to the extracted file system, and rebuild the firmware image.
Although this kit completely automates the process, it is also useful for those who need a handy collection of utilities to manipulate firmware images. The following utilities are included in this kit, along with scripts to automate their use:
ASUSTRX | An extended version of ASUSTRX that can build both 'normal' TRX files and, optionally, those with an ASUS addver style header appended. It can also, uniquely, force segment offsets in the TRX (with -b switch) for compatibility with Marvell ASUS devices like the WL-530g. This tool replaces both 'normal' trx tool and addver. Current versions included are: 0.90 beta. |
ADDPATTERN | Utility to pre-pend Linksys style HDR0 header to a TRX. |
ADDVER | ASUS utility to append a header to a TRX image that contains version information. ASUSTRX includes this capability. Current version: unversioned. |
CRAMFSCK | CRAMFS file system image checker and extractor. Current versions included are: 2.4x. |
MKSQUASHFS | Builds a squashfs file system image. Current versions included are: 2.1-r2, 3.0. |
MKCRAMFS | Builds a cramfs file system image. Coming in next version. Current versions included are: 2.4x. |
MOTOROLA_BIN | A utility that prepends 8 byte headers to TRX images for Motorola devices WR850G, WA840G, WE800G. Current version: unversioned. |
UNCRAMFS | Alternate tool to extracts a cramfs file system image. Use cramfsck instead whenever possible as it seems to be more reliable. Current versions included are: 0.7 (for cramfs v2.x). |
UNSQUASHFS | Extracts a zlib squashfs file system image. Current versions included are 1.0 for 3.0 images and 1.0 for 2.x images (my own blend). |
UNSQUASHFS-LZMA | Extracts an lzma squashfs file system image. Current versions included are 1.0 for 3.0 images and 1.0 for 2.x images (my own blend). Note: Not all squashfs-lzma patches are compatible with one another. I'm working on adding support for all common squashfs-lzma variations. |
UNTRX | Disassembles TRX style firmwares into their component parts. Also supports pre-pended addpattern HDR0 style headers. This was developed exclusively for this kit. Current versions included are: 0.45. |
WRT_VX_IMGTOOL | Utility to generate VxWorks compatible firmware images for the WRT54G(S) v5 series. |
Tools/changes unique to this kit
This kit doesn't just collect existing tools, it also offers new ones and modifications to existing ones. Custom code currently is:
ASUSTRX - Modified to allow '-b' switch to force segment offsets
(needed for some devices, like the WL-530g).
UNTRX - A new tool to extract the contents of a TRX image into its component
parts.
UNSQUASHFS - Custom blend that supports extraction of 2.x squashfs
images. This tool wasn't officially made available until squashfs 3.0, so up
until now it wasn't available for 2.x images.
WRT-VX-IMGTOOL - A new tool to view, extract, build, and fix VxWorks
compatible firmware images for the WRT54G(S) v5 series.
II. Why is this useful?
A list of some of the many reasons this might be desired are:
III. Where to get the kit
The best way to download and use this kit is to check out the sources from the repository. The automated scripts all automatically build the executables, and the sources are the same as you'll find in the tar.gz archive.
To anonymously check out the firmware-mod-kit:
mkdir firmware_mod_kit cd firmware_mod_kit svn checkout svn://svn.berlios.de/firmwaremodkit/trunk |
Kit downloads and links:
Project hosting and repository:
https://developer.berlios.de/projects/firmwaremodkit/
Anonymous svn checkout: svn checkout svn://svn.berlios.de/firmwaremodkit/trunk
Latest official build:
http://www.bitsum.com/files/firmware_mod_tools.tar.gz
/w pre-built linux x86 binaries:
http://www.bitsum.com/files/firmware_mod_tools_prebuilt.tar.gz
Sometimes newer snapshots are posted for testing. When available, they are
here.
User forums can be found at
http://www.bitsum.com/smf.
Packages:
All of the many OpenWrt packages should work with OpenWrt images and many other firmware images.
A specific collection of DD-WRT packages for use with the kit:
http://www.bitsum.com/files/ddwrt_packages.tar.gz .
IV. Requirements
I've worked to make sure the requirements are low enough for anyone to use this kit. However, to reduce distribution costs and better represent the multi-platform capabilities of this kit, the binary tools in the kit do automatically build themselves the first time you run one of the scripts. Any machine set up to build anything C or C++ on the machine will probably be ready. The few requirements are all basic items that everyone has easy access to:
For OS X, installing the Mac OS X Developer Tools will supply the needed GNU C and C++ compilers and make utility.
Any TRX style firmware image format should that uses squashfs-lzma as a root file system work fine. Firmwares that use regular zlib squashfs or unsupported file systems will work if the steps that extract the file system and rebuild the file system are tweaked.
The kit can build firmware images for the following devices, amongst many others:
The kit is known (or thought) to be compatible with the following firmwares:
DD-WRT | tested - versions v23 SP1 and later are compatible (soon older versions too). WARNING: Builds of DD-WRT later than 08/04/2006 (v23 SP2) have some protections against changing the web UI. |
OpenWrt White Russian | untested (probably not atm) - not really necessary, based on OpenWrt has its Image Builder. |
OpenWrt Kamikaze | untested (should work) - not really necessary, based on OpenWrt has its Image Builder. |
FreeWrt | untested - should work ok |
HyperWrt | untested |
Ewrt | untested |
Sveasoft Alchemy | untested |
Sveasoft Talisman | untested |
Linksys / other vendor | not supported by scripts yet - haven't added cramfs handling |
WL-330G (vendor) | untested - should work ok |
WL-520G (vendor) | untested - should work ok |
WL-530G (vendor) | supported |
WL-550G (vendor) | untested - should work ok |
many others* | untested |
* Even if the kit doesn't automatically support the firmware or device you want, you can manually use the tools inside to manipulate many firmware images, or request support be added.
VI. Instructions
I've scripts that simply and abstract the firmware modification process. Basically, it's a
one-step process to extract the firmware, and a one-step process to rebuild the
firmware. There are also scripts to install or remove individual packages or
entire directories of packages.
The procedure of calls is:
A. Extracting the firmware (extract_firmware.sh)
Simply run extract_firmware.sh with the following parameters. You must run this tool from inside the directory it exists in.
$ ./extract_firmware.sh firmware.bin working_directory/
- firmware.bin is the firmware image you want to extract, i.e. DD-WRT.v23_MICRO_GENERIC.BIN.
- working_directory is the working directory you want to use. This is where the intermediate files and the extracted file system will be stored. Note that files in the working directory may be deleted!
It doesn't matter which firmware image you supply, i.e. for a WRT54G or an ASUS WL-500G. These images are usually all the same and differ only in the header format. The rebuilding process will create images for the various models.
An extraction log is created as extract.log.
B. Making changes to the firmware's file system
Modifying the firmware is simple. The file system is stored in the working directory you supplied to extract_firmware.sh. Here are the subfolders of this directory:
rootfs/
This is where the file system is. Make modifications here.
image_parts/
This is where intermediate files are stored. If you need to replace the kernel (not at all recommended), you can do so by replacing the appropriate file here (usually segment2).
installed_packages/
If you use the ipkg_install.sh tool, this is where packages you've installed after firmware extraction will reside.
One can manually change the files in rootfs and/or use the automated package scripts.
Installing packages (.IPK)
In compliment to manually making changes, one can use pre-built IPKG format package files with the ipkg_install/remove scripts. Although not all will work and a repository for packages verified to work has not yet been produced, many packages work just fine.
Packages are pre-built collections of files pertaining to a set of software. OpenWrt and DD-WRT both use packages, and most are cross-compatible. These packages are stored in a tar/gzip archive of a pre-defined structure that includes some control files. You can extract and copy the files manually, or use the ipkg_install.sh script included with this kit. For a list of some of the available packages (not all may work), see http://downloads.openwrt.org/whiterussian/packages/ and http://ftp.selfnet.de/pub/openwrt/backports/rc5/ . We are currently in the process of compiling a set of packages known to be good for DD-WRT ... to help in this matter please visit here.
For information on how to create IPKs, see the IPK Creation Kit.
Example use:
$ ./ipkg_install.sh some_package-1.2.5.ipk working_directory/
some_package-1.2.5.ipkg would be the filename of the package.
working_directory is the same working directory you supplied to the extract_firmware.sh script.Notes:
A script, ipkg_install_all.sh is also available to install an entire folder of packages.
The collection of installed packages will be stored in working_directory/installed_packages .
Removing packages (.IPK)
To remove packages there is the ipkg_remove.sh script.
Example use:
$ ./ipkg_remove.sh some_package-1.2.5.ipk working_directory/
some_package-1.2.5.ipkg would be the filename of the package.
working_directory is the same working directory you supplied to the extract_firmware.sh script.Notes:
A script, ipkg_remove_all.sh is also available to install an entire folder of packages.
The collection of installed packages stored working_directory/installed_packages will be updated to reflect the change if successful.
C. Re-building the firmware (build_firmware.sh)
Rebuilding the firmware is as easy as extracting it. Use the build_firmware.sh script to automate the process. You must run this tool from inside the directory it exists in.
Example use:
$ ./build_firmware.sh output_directory/ working_directory/
- output_directory is the path to which the created firmware images should be stored. If images already exist here, they will be over-written. Firmware images for various models will be emitted (these images are all the same but have different header patterns so they are accepted by the target models).
- working_directory is the working directory supplied to extract_firmware.sh.A building log is created as build.log.
VIII. Caveats
There are a few things to watch out for when modifying firmware images. If you make reasonable changes these problems will probably never affect you. Be warned, this section may be technical in parts.
A. Binary compatibility issues
Currently, the maximum firmware image size supported by this kit is 5,898,240 (0x5A0000) bytes, but can be adjusted through the "-m [size]" parameter of the asustrx tool.
C. Solutions to file system
incompatibilities
! This section is very much a work in progress !
The squashfs file system is case sensitive (i.e. teSt.bin ! = test.bin) and adheres to EXT2 naming rules. To support proper extraction of the file system, the working directory should be on a case sensitive file system and support symbolic links. The scripts will warn you if you attempt to use as working directory on a case INsensitive file system. You'll see a number of errors if you extract to a file system that doesn't support symbolic links.
Note: Case insensitivity doesn't always cause problems, especially if the file system preserves case ok. It depends on the firmware and version. Most of the time for DD-WRT as long as a case preserving file system is used as a working directory, all should work fine.
These solutions apply to those using CYGWIN, or some other environment that may not have a case sensitive file system by default. Linux users typically do not have this problem since EXT2 and EXT3 file systems are case sensitive.
WARNING: THESE ARE IN EARLY STAGES OF TESTING AND MAY NOT WORK.
Applies to | Description | |
SOLUTION 1 | CYGWIN, OTHER | It's reported this does not work under
CYGWIN due to the CYGWIN utilities all being case INsensitive
(thanks pagedude). Use a network share to a case sensitive file system. This is, by far, the easiest solution. Simply use an appropriate network share for the working directory. |
SOLUTION 2 | CYGWIN | For NT partitions it's reported you can enable case sensitivity through the following registry key, but I do not know if this works or not. \HKLM\SYSTEM\CurrentControlSet\Control\Session
Manager\kernel\obcaseinsensitive [REG_DWORD] = 0 |
SOLUTION 3 | OS X (not usually needed, at least for DD-WRT) | One can either Turn on HSFX for the boot disk, or (more easily) create a temporary HSFX case sensitive file system to use as a working directory. Example
of how to create a temporary HSFX case sensitive file system: (thanks solarflare for this information) |
SOLUTION 4 | CYGWIN |
Mount an ext2 files system. Eko, of DD-WRT forums, contributes this tip. Make a partition on a hard drive or USB thumb drive, then install "Ext2 Installable File System For Windows". 10MB should generally be plenty of working directory space, but 20MB is suggested for more optimal file system performance. |
IV. Acknowledgments and more
The author and maintainer of this kit is Jeremy Collake (db90h). Special thanks to Solarflare, TexHex <floh@fbesser.de>, and others.
If YOU would like to contribute to this kit, send me an email at jeremy.collake@gmail.com or visit the current project hosting at http://code.google.com/p/firmware-mod-kit/ . Developers are welcome.
X. Links
0.51
0.50
0.49
0.48
0.471
0.47
0.46:
0.45:
0.43:
0.42:
0.40 beta (md5: 786472cfa03fb1cd70e1b025d091168b)
0.33 beta
0.31 beta
0.30 beta
0.29 beta
0.28 beta
0.27 beta
0.26 beta
0.25 beta
0.23 beta
0.22 beta
0.20 beta
v0.15 beta
v0.13 alpha: Documentation updates, re-included pre-built executables
v0.10 alpha: First decent public release.
XI. About the author
I've been doing R&D on for the WRT community for a few months now. Notable accomplishments include developing a method to flash linux onto the WRT54G(S) v5 series without hardware modifications, an assortment of utilities including one to manipulate VxWorks compatible firmware images for the WRT54G(S) v5 series, one to manipulate default nvram variables embedded in CFEs, one to encode and decode sdram parameters, and many more. I've been a commercial software developer for over 10 years, specializing in low-level programming for the x86 architecture, primarily for Windows. My web site is at http://www.bitsum.com, where you'll find links to some of my software, including PECompact, a popular Win32 executable compressor.
Contact me at jeremy@bitsum.com if you need work done or or donate money to encourage more works like this by donating via Paypal to jeremy@bitsum.com. There are also other ways to help. Feel free to contact me if interested.
This document (c)2006,2007,2008 Jeremy Collake.
All Rights reserved. This document may be freely republished in its unaltered
and whole form only. Alterations or partial publishing requires approval of
Jeremy Collake <jeremy@bitsum.com>.
This kit is free software; you can redistribute it
and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA