Module System

Introduction

TODO this is a work in progress, please help fill it in.

Starting with version 1.1, CHDK uses loadable binary modules (similar to .dll or .so files on other platforms). This reduces memory requirements and allows platform independent binaries to be made.

Development thread http://chdk.setepontos.com/index.php?topic=847.0

Thanks to tsvstar for the initial implementation of the module system.

For end users

In 1.1 and later, a minimal install of CHDK must include the modules as well as well as the CHDK core (DISKBOOT.BIN and/or PS.FI2, PS.FIR). Modules reside in CHDK/MODULES and have a .flt extension.

On multi-partition cards, the modules must be installed in the main (large) partition that the camera sees, not the small boot partition with diskboot.

For developers [TODO]

---

Common module processing sequence

1. Module loaded using function: module_load( char* name, _module_bind_t callback).

   name = module name (path relative to A/CHDK/MODULES/ or from root if started from '/')
   callback = callback function to bind/unbind module symbols to CHDK

   1. Allocate space, load module, made relocations
   2. Check module format type and version to strict correspondence. Any load is canceled
   3. Replace imported by modules symbols with real adresses. If no such symbol (by hash) found in CHDK core list, load is canceled
   4. Call callback() if it is defined.
      Purpose of this function - a) bind exported by modules symbol to core variables; b) Check all specific to module and required by core prerequisites. If something wrong module loading could be canceled by returning value !=0.
   5. Call _module_loader() function of module if it is exist.
      Purpose of this function is check prerequisites (such as API versions for example) and make module initialization. If something wrong (fail compatibility of required API version check, for example) - return !=0 and module loading will be canceled (at this stage callback(0) will be called to unbind this module).
2. Module usage.
   • Module could have _module_run() function. This is main entry point of module and it is used to standalone and mixed modules. It is called with variable length list of arguments and returned value is returned at call point
     Simple syntax (standalone modules) to call module main entry is rv =module_run("module.flt", callback, sizeof(argv)/sizeof(argv[0]), argv, UNLOAD_IF_ERR);
     Note: If module support case with empty argument list (argn=0, argv=0) then it could be ran with file browser and dynamic menus. Otherwise it could be called only if such call is defined in the CHDK code.
   • Module could be raw library. Then CHDK core decide when and which one imported from modules symbols should be called.
3. Module unload. When module not needed anymore it should be unloaded to free memory and module slot (limited number of modules could be loaded by CHDK at same time).
   Important note: Please be careful with load/unload. A lot of possible nuances exists. For example, owned by module resources could be used somewhere else (LUA), or should exists until shutdown (edge map), etc.
   • Unload sequence is:
     1. Call _module_unloader function of module if it is exist.
        Purpose of this function free allocated by module resources
     2. Call core unbind callback function (parameter "callback" of module_load/module_run).
        Purpose of this function - unbind modules symbols from variables of the core of CHDK
     3. Unallocate module
   • How module could be unloaded:
     • Just unload.
       Caller (module or core) could unload loaded module if it is not needed (Using module_unload()/module_unload_idx()).
     • Asynchronous unload.
       Module could unload itself when it is finished. Because it is unsafe to unallocate module while execution control flow is inside it, this happens asynchronously.
       How to do this: call module_async_unload() right before end of function. Do this only if right after this execution flow will leave module and you are sure that it will not returned back to module without module_load() call before.
     • Automatical unload on finish run handler.
       If module was ran with flag UNLOAD_ALWAYS it will be unloaded right after finishing its _module_run callback.
     • Automatical unload on press <ALT>.
       GUI mode could be switched to none at any moment of time by pressing <ALT> button. This event cause automatical unloading of all loaded modules. Because module doesn't know that it lost focus and probably will not get it again.
       But sometimes such behaviour is not desirable (need to keep resources till shutdown; this is library and should be unloaded by caller; unsafe because caller wait finishing with callback; etc). To prevent such kind automatic use module_set_flags(module_idx, MODULE_FLAG_DISABLE_AUTOUNLOAD);

Module templates

How to add new module

API versioning system

Communications between modules is some kind of API's. It include exact set of functions, data structures and rules how to use them.

For case of static link we can be sure that everything is aligned one to another because we are sure about all modules versions.
For case of dynamic linking (which FLAT modules are), we need to have a way to make sure that module have required (or compatible) API. This is what API versioning system is purposed for.

Base rules of API versioning system:

• Any data structure or function list should have shared variable which describe it's API version. This could be separated variable or first(!) member of this structure.
  Examples: core CHDK system-indenpendent structures, dynamic libraries modules (shared API structure), ...
• Several different structures could be identified as one API for version system to decrease number of mandatory checks (if this is common structures).
  Examples: gui_version.common_api - describe (and so should be changed if any) versions of camera_screen, gui_handling, mbox. This components are used almost everywhere and so it is good idea to check only one version instead of several.
• Any change in related API always should affect to its version. No exclusion allowed.
  If changes are backward compatible (new member added to the end, new function added) - increment API minor version. Try to keep this way.
  If changes are backward incompatible (changes in the middle of structure, changes in the meaning of some values, changes in rules how API works) - increment API major version and zero minor version.
  
  Why this is so important:
  1. API VS is only way to detect what real binary format is and how API work. Changed version will prevent incompatible data usage and function calls.
     
  2. API version is not marketing value and visible nowhere, so it easily could have any big values. This is only way to separate one API from different one.
     
• Before any API usage, its version should be checked. Check it once during binding is enough. API_VERSION_MATCH_REQUIREMENT( api_ver_variable, REQUIRED_MAJOR_VER, MINIMUM_MINOR_VER )
  - return true if major version is equal to required and minor version is greater or equal to required.

Core CHDK API:
camera_sensor.api_version
camera_info.api_version
Conf.api_version
gui_version.common_api
gui_version.menu_api