Adding Firmware Features

Finding firmware function entry points

Functions whose name is not found in strings

Canon engineers are like the rest of us, they don't like to reinvent the wheel. Camera models share a lot of code fragments and they are likely compiled with versions of the same compiler tools. This results in similar assembly code. Thus, if a function has been found for one camera, it's likely that similar functions may be found in other cameras by searching parts of the function code from binary code or disassembly. This is what the automatic signature search in CHDK source tree tools directory does. A full binary match is typically not found, because RAM and ROM addresses usually change, and register selection may change as well.

For functions which don't seem to have any strings attached and that haven't been found in any camera... well, too advanced for this guide. I suppose being a native speaker in ARM assembler helps and being good at making educated guesses based on experience in knowledge of other parts of the firmware, other embedded software projects and ARM C compilers isn't going to hurt...

Functions whose name is found in a string

With a "string" I mean something that's pretty much encoded in clear text in the firmware dump binary. For example the gcc toolset command 'strings -t x PRIMARY.BIN >strings.txt' will give you that, equipped with a start address of each string in hex.

1. Find a string that interests you
2. Look at the disassembly to see where in the code the string is used. Go there.
3. Does that look like a function? Does someone call it? If yes and yes,
4. Try calling it yourself (if you're fearless), or try looking a what it does to figure out if it should be safe to call it and then call it :)

In some cases the "function" which loads the string is just two instructions: one to load the string and one to jump to the actual function. There may be several pairs like this in a row, forming a "jump table". What you desire can then probably be achieved either by using the address which loads the string, or the next address or the address it jumps to and the only difference will be that skipping the string load instruction makes Canon's debugging system less accurate. No biggie, and the code will be a few clock cycles more efficient.

Note that there are some functions that by their name alone sound pretty dangerous to fiddle with. For example in the a570is 1.00e firmware, strings such as these can be found... it's probably for the best not to jump to these without good reason unless you don't care for the well being of your camera:

ffc08ae4 WriteToRom

ffc14514 ExecuteResetFactoryWithRomWrite

And these just can't go unmentioned:

ffcedc18 MANGO_PUDDING

ffd63cd0 ChangeImagerToMangoPudding

ffe02708 ChangeImagerToMangoPudding

ffe18720 ChangeImagerToMangoPudding

ffe8d670 Sharapova->%d

Now that you've been warned, let's continue with an example. I had a problem. The Ev compensation setting does not affect the LCD live preview image unless you are half shooting or currently adjusting Ev compensation. Normally this is not a problem, but during motion detection it may sometimes be quite essential (unless running motion detect under half shoot).

In the firmware dumps (I checked a570is, a720 and s3is) there are two strings that caught my attention (pasted from a570is 1.00e strings dump):

ffea4f64 SSAPI::EnterToCompensationEVF

ffea4fe8 SSAPI::ExitFromCompensationEVF

EVF is short for "Electronic View Finder". If these are functions and if they can be called at will, from their names I would assume they would do exactly what I want. If that's true, we could either add script commands to call these functions or modify md_detect_motion code to always call it.

The task at hand is to find a way to call these functions, because so far we only know where their names are mentioned. Typically a function starts by a push (stmfd) instruction saving the value of some registers to the stack, with lr (link register) being usually among them and ends by a pop (ldmfd) instruction to bring them back.

What causes the function to actually return is writing the return address to the pc (program counter) register. Code that calls functions usually does so using the bl (branch with link) instruction, which stores the return address to the link register (lr). Thus a function returns by writing lr to pc. But most functions also jump to subroutines and often do so with bl, which obviously overwrites lr. That's why many functions need to push lr to the stack to save it from getting lost. Then, when it's time to return, it just pops this value straight to pc.

If the function has arguments or return values, they are often passed in registers as well, and these ones are not pushed/popped. r0 to r3 are are usually used for parameter/return value passing (if there are any).

Between functions there are so called "literal pools" of instructions that aren't really instructions. They are addresses and constants. They are placed between functions, because the ldr instruction can only fetch them from a 1024 word range (up or down) in the ROM. That makes things a bit easier to read since most addresses can be found nearby. Even strings of function names are often next to the functions themselfs for this reason.

Let's look at the disassembly (see Gpl_Disassembling):

; The string "SSAPI::EnterToCompensationEVF", written in 8 words 
; (50 41 53 53 = P A S S, turn it around and you get SSAP). Disassemby on these
; rows is garbage, because they are never executed as instructions:
ffea4f64:       50415353        subpl   r5, r1, r3, asr r3
ffea4f68:       453a3a49        ldrmi   r3, [sl, #-2633]!
ffea4f6c:       7265746e        rsbvc   r7, r5, #1845493760     ; 0x6e000000
ffea4f70:       6f436f54        svcvs   0x00436f54
ffea4f74:       6e65706d        cdpvs   0, 6, cr7, cr5, cr13, {3}
ffea4f78:       69746173        ldmdbvs r4!, {r0, r1, r4, r5, r6, r8, sp, lr}^
ffea4f7c:       56456e6f        strbpl  r6, [r5], -pc, ror #28
ffea4f80:       00000046        andeq   r0, r0, r6, asr #32
; Load string pointer address to r1 (argument for the next subroutine call):
; pc, #80 == pc+80 == ffea4fdc. It's enclosed in [], which means indirect
; addressing i.e. that r1 is not loaded with ffea4fdc, but instead value in
; memory address ffea4fdc, which happens to be ffea4f64. And that is the 
; start address to the string that contains the name of this function.
ffea4f84:       e59f1050        ldr     r1, [pc, #80]   ; ffea4fdc VALUE:<ffea4f64> STRING:<SSAPI::EnterToCompensationEVF>
; initialize r0=0x20 (argument for the next subroutine call):
ffea4f88:       e3a00020        mov     r0, #32 ; 0x20
; we're about to jump to a subroutine so we need to save the return address:
ffea4f8c:       e52de004        push    {lr}            ; (str lr, [sp, #-4]!)
; gosub (this is probably an error check/reporting function of some sort):
ffea4f90:       ebf58242        bl      ffc058a0 <_binary_dump_bin_start+0x58a0>
; gosub SSAPI::HardwareError
ffea4f94:       ebfffa76        bl      ffea3974 <_binary_dump_bin_start+0x2a3974>
; if r0!=0 then return 
ffea4f98:       e3500000        cmp     r0, #0  ; 0x0
ffea4f9c:       149df004        popne   {pc}            ; (ldrne pc, [sp], #4)
; gosub SSAPI::LowBattery
ffea4fa0:       ebfffa82        bl      ffea39b0 <_binary_dump_bin_start+0x2a39b0>
ffea4fa4:       e3500000        cmp     r0, #0  ; 0x0
; If battery low then return. This ldr fetches to r1 a RAM address 0x000a5788, could be interesting.
; This function appears to have 3 return values (ram address 0xa5788 via r1 and two numbers in 
; r0 and r2 but we can probably just ignore them.
ffea4fa8:       e59f1030        ldr     r1, [pc, #48]   ; ffea4fe0 VALUE:<000a5788>
ffea4fac:       e3a02002        mov     r2, #2  ; 0x2
ffea4fb0:       e3a000cf        mov     r0, #207        ; 0xcf
ffea4fb4:       149df004        popne   {pc}            ; (ldrne pc, [sp], #4)
; gosub
ffea4fb8:       ebf59b35        bl      ffc0bc94 <_binary_dump_bin_start+0xbc94>
ffea4fbc:       e3a01d25        mov     r1, #2368       ; 0x940
ffea4fc0:       e3100001        tst     r0, #1  ; 0x1
ffea4fc4:       e2811005        add     r1, r1, #5      ; 0x5
ffea4fc8:       e59f0014        ldr     r0, [pc, #20]   ; ffea4fe4 VALUE:<ffea3288> STRING:<ShootSeqAPI.c>
ffea4fcc:       0a000000        beq     ffea4fd4 <_binary_dump_bin_start+0x2a4fd4>
ffea4fd0:       eb0095f6        bl      ffeca7b0 <_binary_dump_bin_start+0x2ca7b0>
; bring back return address to lr and jump somewhere and never return to this part of code...
ffea4fd4:       e49de004        pop     {lr}            ; (ldr lr, [sp], #4)
ffea4fd8:       ea000fff        b       ffea8fdc <_binary_dump_bin_start+0x2a8fdc>

Now we could go on and follow that branch but it already looks like this function does some error checks, then something more productive and that it is taking care of keeping the return address safe, so it's probably fair to assume that jumping to ffea4f84 as an address for SSAPI::EnterToCompensationEVF could work (it's the address of the first instruction after the string, and the functionality seems solid from there on). And it gets better: when searching for ffea4f84 in the disassembly, three bl instructions branching to this address can be found. So, apparently this is a subroutine used by the firmware.

In exactly the same way a start address for ffea5008 SSAPI::ExitFromCompensationEVF can be determined.

We still have no idea what happens when we call these. The next step is to add these functions to CHDK.

Adding firmware functions to CHDK

This guide gives an example of adding two firmware functions to CHDK and implementing an uBASIC command which calls them. A "firmware function" here means a subroutine which already exists in the camera firmware. This guide assumes you know the correct entry point (start address in ROM) of this function. We are not changing the firmware function in any way, just calling parts of it and hoping this is useful and harmless.

We now have what we think are entry points for two functions of the a570is 1.00e firmware:

0xffea4f84 SSAPI::EnterToCompensationEVF

0xffea5008 SSAPI::ExitFromCompensationEVF

They can easily be called from CHDK as long as we give them names and point the compiler to their locations. Much of CHDK's operation already depends on Canon's firmware functions called this way, so everything is already readily set making this quite easy.

I will loose the SSAPI:: and name these functions EnterToCompensationEVF and ExitFromCompensationEVF in CHDK. I will then add a new uBasic command set_comp_evf to call them from a script.

It's a good idea to check all stub files (platorm/*/sub/*/*.S) for matches to prevent unnecessary work in case the stubs are already there.

1) Add the functions to platform/a570/sub/100e/stubs_entry_2.S:

NHSTUB(EnterToCompensationEVF, 0xFFEA4F84)
NHSTUB(ExitFromCompensationEVF, 0xFFEA5008)

NHSTUB is a macro defined in stubs_asm.h, which transforms each of those two lines into a global inline assembler function (_EnterToCompensationEVF and _ExitFromCompensationEVF) that jumps to the specified address.

2) Add prototypes to include/lolevel.h:

extern void _EnterToCompensationEVF(void);
extern void _ExitFromCompensationEVF(void);

3) Add wrapper functions to platform/generic/wrappers.c:

void EnterToCompensationEVF(void)
{
  _EnterToCompensationEVF();
}

void ExitFromCompensationEVF()
{
  _ExitFromCompensationEVF();
}

4) Add prototypes of the wrappers to include/platform.h:

void EnterToCompensationEVF(void);
void ExitFromCompensationEVF(void);

The reason that we need these wrapper funtions is that CHDK code is mostly (core and lib) running in the ARM thumb mode to make it smaller. These Canon functions however run in ARM mode. The platform code of CHDK runs in ARM mode, which is why we place these new functions there. The wrappers provide thumb mode interfaces to the ARM mode functions.

We now have two functions CHDK can call from core and lib code using the syntax:

EnterToCompensationEVF();
ExitFromCompensationEVF();

I'm not saying calling these is safe or that they do anything; just that they can be called and CHDK will build and if called, it will execute them.

5) We now have functions but nobody calls them yet. At this point you're of course eager to test out your new function. So let's make a quick test by temporarily modifying an existing uBasic command. I selected the shut_down command and modified it to call EnterToCompensationEVF() instead of camera_shutdown_in_a_second() by modifying lib/ubasic/ubasic.c:

static void shutdown_statement(void){
  accept(TOKENIZER_SHUT_DOWN);
//  camera_shutdown_in_a_second();
  EnterToCompensationEVF();
  accept_cr();
}

Then I wrote a simple uBasic script to test the command:

rem test the shut_down command
@title shut_down test
@param a dummy
@default a 0
sleep 2000
shut_down
sleep 2000
end

I compiled chdk, installed the new build and the above script on a card.

Then I started my camera in P mode, set Ev compensation to -2 to clearly see if the LCD got darker. Pressing the +/- button toggles the LCD brightness by 2 Ev stops. And yes, running the script forces LCD to be dark even when the Ev compensation setting is not being edited. Success!

Looks like the function indeed works, but of course not in M or Video modes since there is no Ev compensation. Half-shooting or shooting does not reset the EVF back, but turning the mode dial or changing SCN submode does.

6) With the modifications above, CHDK will only build for cameras which have stubs for these functions defined. This must obviously be fixed.

Enclosing much of what was added in #ifdefs should work, but it will make the code ugly. Alternatively, dummy stubs could be used for cameras which don't have these functions (not yet found or not existing at all). This is a nice way, because when the function is found for that camera, it's easy to just replace the dummy address in the stub with a real one, and the sources will clearly indicate that a feature is missing for that model if someone reviews that file. However, not all platforms have suitable dummy stubs ready for us to use. So let's just add a commented-out stub and add a dummy function in lib.c instead.

For all firmwares you haven't found the subs, add the following dummy functions to lib.c:

void _EnterToCompensationEVF() {} // Dummy function. To be removed after stub is found. See stubs_entry_2.S.
void _ExitFromCompensationEVF() {} // Dummy function. To be removed after stub is found. See stubs_entry_2.S.

And to be nice, add this to stubs_entry_2.S:

//NHSTUB(EnterToCompensationEVF, 0xFFFFFFFF) // Stub not found. When found, remove dummy function from lib.c.
//NHSTUB(ExitFromCompensationEVF, 0xFFFFFFFF) // Stub not found. When found, remove dummy function from lib.c.

I wrote a simple helper script to make adding these dummy functions to other platforms easier (tested on Linux, should probably work under Cygwin as well): Add_to_all_subs.sh.

Go to your CHDK source tree root directory, place the above stubs code to "stubfile" and the lib.c code to "libfile" and then run the script twice:

add_to_all_subs.sh stubs_entry_2.S stubfile

add_to_all_subs.sh lib.c libfile

This adds lines in stubfile and libfile to all non-zero stubs_entry_2.S and lib.c files (respectively) it can find in platform/*/sub/*/ directories.

The platform(s) which already have your feature were also modified, so you now need to remove these newly added lines from those. After that, you're all set to build on all platforms (you should test that before submitting your code; 'make batch-zip' should work).

Of course the proper way to go is to find stubs for all firmwares you possibly can. For that you need to disassemble all firmware dumps. This is the tedious part as there are dozens of supported firmwares and also because you can't quickly test cameras you don't have access to.

stubfile:

//NHSTUB(EnterToCompensationEVF, 0xFFFFFFFF) // Stub not found. When found, remove dummy function from lib.c.
//NHSTUB(ExitFromCompensationEVF, 0xFFFFFFFF) // Stub not found. When found, remove dummy function from lib.c.

libfile:

void _EnterToCompensationEVF() {} // Dummy function. To be removed after stub is found. See stubs_entry_2.S.
void _ExitFromCompensationEVF() {} // Dummy function. To be removed after stub is found. See stubs_entry_2.S.

---

Adding a command to uBASIC

This is a guide by and example sort of a document for adding a command to uBASIC.

1) Now it's time to add access to these new functions EnterToCompensationEVF() and ExitFromCompensationEVF() to uBASIC for real. It would be a waste of resources to add commands for both functions, so let's just add one command with a single parameter to select whether we want to "enter" or "exit" compensation EVF. I selected the name set_comp_evf, and argument 1 for "Enter", 0 for "Exit". Remember that it's not a good idea to give uBASIC functions humongously long names, because script writers have a maximum script size limit to worry about.

Let's select a suitable existing uBASIC command which has one integer parameter and use it as a template. "sleep" should be perfect. Its functionality lies in lib/ubasic/ubasic.c:

static void
sleep_statement(void)
{
  int val;
  accept(TOKENIZER_SLEEP);
  val = expr();
  ubasic_camera_sleep(val);
  DEBUG_PRINTF("End of sleep\n");
  accept_cr();
}

So, let's add this to lib/ubasic/ubasic.c (somewhere before the function static void statement(void)):

static void
set_comp_evf_statement(void)
{
  int val;
  accept(TOKENIZER_SET_COMP_EVF);
  val = expr();
  if (val>0) EnterToCompensationEVF();  // 1: enable ev compensation on LCD
  else ExitFromCompensationEVF();       // 0: disable ev compensation on LCD
  accept_cr();
}

The above function isn't actually called by uBASIC yet, so we need to add it to the switch/case statement near the end of ubasic.c:

  case TOKENIZER_SET_COMP_EVF:
    set_comp_evf_statement();
    break;

And for that "case TOKENIZER_SET_COMP_EVF" to ever happen, we need to add the function name to lib/ubasic/tokenizer.c:

  {"set_comp_evf",            TOKENIZER_SET_COMP_EVF},

And finally, to lib/ubasic/tokenizer.h:

  TOKENIZER_SET_COMP_EVF,

I didn't specify line numbers or give diffs, because these files are under constant development. It should be quite easy to figure out where those modifications go. If it isn't, the guide is either outdated or you probably shouldn't be doing this ;).

2) In addition to these there are two script related files, camera_functions.c and camera_functions.h, which have functions for simulating scripts outside the camera. This new command should probably be added to those as well, but I'm not sure how. Not touching those files shouldn't affect CHDK running in a camera, and if one wants to simulate a script with our new set_comp_evf command, the simulation will succeed if that command is just removed or replaced with "sleep 1".

3) One more uBasic test script to try out our newly added command:

rem test the set_comp_evf command
@title set_comp_evf test
@param a dummy
@default a 0
set_comp_evf 1
sleep 1000
set_comp_evf 0
sleep 1000
set_comp_evf 1
sleep 1000
set_comp_evf 0
end

And it works beautifully!

4) Write documentation for the new command:

Syntax: set_comp_evf a.

If a==1, calls Canon firmware function EnterToCompensationEVF.

If a==0, calls Canon firmware function ExitFromCompensationEVF.

Other values of 'a' are reserved, do not use them in scripts.

The Canon firmware function EnterToCompensationEVF causes the LCD viewfinder image to adjust according to the user settable Ev compensation setting. ExitFromCompensationEVF makes LCD not to adjust with the Ev compensation setting.

Normally the camera (verified on a570is, likely applies to all or most other models) is in the CompensationEVF mode only while the shutter is half pressed or while editing the Ev compensation setting.

Note that the Ev compensation setting is not available in all shooting modes (on a570is, the manual "M" mode and the video modes). Consequently, this function seems to have no effect in those modes.

Turning the mode dial or switching to play mode apparently resets the effect of this function, but shooting (or half-shooting) doesn't.

Adding a command to Lua

Adding commands to Lua is easy: Just add a function luaCB_NameOfYourLuaFunction() to modules/luascript.c and then add FUNC(NameOfYourLuaFunction); to register_lua_funcs() function. We already have a large number of Lua functions with different sets of input arguments and return values, so there are lots of good examples available.

Our EVF example requires two modifications to modules/luascript.c:

Add a new function:

static int luaCB_set_comp_evf( lua_State* L ) 
{
  int val = luaL_checknumber(L, 1);
  if (val>0) EnterToCompensationEVF();  // 1: enable ev compensation on LCD
  else ExitFromCompensationEVF();       // 0: disable ev compensation on LCD
  return 0;
}

And to function register_lua_funcs(), add the line

FUNC(set_comp_evf);

This FUNC() is a macro which is defined in the beginning of this function, which causes luaCB_set_comp_evf() to be tied to set_comp_evf() command in Lua.

All done! To use the command, just put set_comp_evf(1) or set_comp_evf(0) in a Lua script. And like for uBASIC, don't forget to document your functions to the Wiki!

Finding RAM addresses

Useful RAM addresses are often found while reading a disassembly in ldr instructions that indirectly load values typically below 0xfffff to a register. If searching for something particular, checking disassemblies of functions found by searching for strings that seem to relate should be a good place to start.

You can take a look at the behavior of a RAM address using the built-in CHDK memory browser or for example by modifying the CHDK built-in "debug misc values" display in gui.c to show the contents of RAM addresses you are interested in:

In function void gui_draw_osd(), find if (debug_vals_show) { ... and change one of the sprintf lines, for example:

//sprintf(osd_buf, "1:%8x  ", physw_status[0]);
sprintf(osd_buf, "1:%8x  ", *((unsigned int*)0xa30c8));

The first line will now show the value of RAM address 0xa30c8 on your LCD if debug "show misc values" is enabled.

Adding functions for camera RAM variables

1) Add the RAM location stub to platform/a570/sub/100e/stubs_min.S:

DEF(movie_status, 0xA30C8)  

DEF is a macro defined in stubs_asm.h, which transforms this line into a global inline assembler call which returns the desired value.

2) Add variable declaration to include/platform.h:

extern int movie_status;

You should now be able to read and write the movie_status variable in CHDK core if platform.h is included. And the obligatory warning: The fact that this is possible doesn't mean it's safe.

3) If you now add some code that uses your variable, CHDK will only build for a570is 1.00e. You need to find the addresses and add stubs for all platforms or define dummy variables with constant values to make CHDK build for other cameras as well.

Again, the Add_to_all_subs.sh script helps a bit here.

Go to your CHDK source tree root directory, create a file called "stubfile" with the following contents:

// DEF(movie_status, 0xFFFF) // Stub not found. When found, remove dummy variable definition from lib.c.

Create a file called "libfile" with the following contents:

int movie_status = 0; // Dummy variable definition. To be removed after stub is added to stubs_min.S.

Then run the add_to_all_subs.sh script twice:

add_to_all_subs.sh stubs_min.S stubfile
add_to_all_subs.sh lib.c libfile

This adds lines in stubfile and libfile to all non-zero stubs_min.S and lib.c files (respectively) it can find in platform/*/sub/*/ directories.