GUIDE TO THE SOURCE CODE OF SIMPHONE 0.9.2

1. Directory Structure

2. qsimphone

The first few lines of each .cpp file contain a comment describing which classes are implemented by that source file and what purpose they serve.

Each source file usually has an associated .h file of the same base name, which defines those classes. There are some comments that describe class members there.

The rest of the files (.ui) contain Qt resources (definitions of GUI elements in XML format).

The flags subdirectory contains all country flags in PNG format.

The icons subdirectory contains all other icons used by the GUI, mostly in PNG format.

The translations subdirectory contains translations of GUI messages to different languages in XML format (readable by Qt Linguist).

The resources subdirectory contains the IP address database and translated GUI messages in binary format.

When changing the source code of the GUI or simcore, please follow the existing coding style. clang-format-9 must be run on the source code to format it properly.

3. simcore Overview

The first two lines of each .c file contain a comment describing the purpose of that module. Each module usually has an associated .h file of the same base name, which defines functions exported by the module, as well as data structures (if any). Structure fields are commented; most of the function definitions also include a short comment describing the purpose of each function.

Interaction between the GUI and simcore is done through the simcore public API (please refer to api.html for a description).

The .h files define the internal API of simcore; it is used for interaction between different modules. Functions are called to perform actions, while data structures are passed as arguments to those functions and are used to pass data between different modules. Usually only the module which defines a structure has the job of writing to it. In object-oriented terminology, some of these structures would define the "classes" of simcore.

4. Special simcore Modules

spth.c and nPth

The nPth library used by simcore provides pseudo-threads (which are really coroutines) using real (preemptive) threads, as provided by the operating system. It implements functionality of another library called Pth, but not its API. In order to allow simcore to use either nPth or Pth, the glue code in spth.c implements some of the Pth API, using nPth as an implementation.

Additionally, it provides access to the npth_protect and npth_unprotect functions (not available under Pth) and brings the Pth API in line with the simcore "underscore" naming convention (described under The simcore Naming Convention), for better code readability.

Multiple threads are used by simcore to perform various tasks, but only a single thread can usually run at a given time; this happens with both the single-threaded and the multi-threaded version. While this may seem an unnecessary restriction, it allows to dispense with complicated synchronization between threads and avoids risks of unforeseen race conditions and deadlocks. simcore threads usually perform only control tasks that do not take a lot of CPU time; a thread is suspended automatically by nPth/Pth when it has to wait for an event (such as receiving data from the network), while other threads can run in the mean time. simcore threads can also allow other threads to run at the same time, while they are performing a CPU-intensive task (such as a TLS handshake), thus taking advantage of a multi-core CPU. This is done by calling pth_unprotect and pth_protect_ explicitly, two functions which do nothing in the Pth (fully single-threaded) version. One can still enable time-sharing for the Pth version by calling pth_thread_yield_.

You can think of nPth/Pth as a magic device that turns a multi-threaded program into a single-threaded select loop at run-time (and in case of Pth, it really uses select and does not use more than one system thread).

proto.h

While not really a module, this header file defines all SIMTABLE keys of all simphone network protocols. A comment for each key describes the meaning of its SIMTABLE value.

const.c

This pseudo-module includes all .h files that contain simcore resources (large read-only byte arrays, such as built-in sounds).

crypto.cpp

The only special thing about the crypto module is that its written partly in C++, as it has to interface to the crypto++ library, and makes heavy use of templates. The rest of simcore is written in pure ANSI C.

Note that the crypto++ library uses static constructors; this means that linking to simcore will insert "invisible" function calls that are invoked by your program prior to entering its main function.

console.c

The console module contains code that uses mostly the simcore public API, as most of the module runs at UNLOCK level. It provides an example of how to call the API from C.

table.c

Implementation of simtypes. The sim_table_read and sim_table_write functions serialize simtypes (convert them from/to a byte sequence) and take an opaque pointer to a user-defined context structure which simstreamer callbacks can access.

simc, simsize, simdump and simxmls

These are in the simcore subdirectory, but they are really stand-alone programs, not simcore modules.

simc is the main testing program for simcore. It uses the console module and has batch capability, but not scripting capability, because it lacks an interface to any scripting language. You can use it to execute "scripts" consisting of multiple console commands either at startup or at a later time (on receipt of a unix signal). The GUI has the same capability via the qtsingleapplication RPC (also available on Windows), but for security reasons, this is enabled by default only in the "debug" build.

To compile simc, execute:

   cd simcore
   make simc

Most simphone functions are available with simc, so it can be abused as a poor man's Simphone in environments where the GUI cannot be used.

simxmls is a command-line program that allows you to manually merge multiple chat history files, at the same time cleaning them up by skipping duplicate, corrupted and removed messages. Edited messages remain in place (the original text as well as any and all edits of it).

module hierarchy

There is no clear hierarchy of simcore modules, as many of them are inter-dependent. However, modules can be logically grouped in the following (approximately bottom-to-top) way:

1. basic services:
   • utilities: spth, logger, table, utils, system/firewall
   • cryptographic implementation: crypto/aria, ssl
   • cryptographic services: file, socket, keygen
   • network and file services: network, mainline, contact, param
2. network protocol: limit/proxy, proxies, nat, server, client
3. network users: msg, xfer, audio
4. simcore interface: api
5. simcore users: console, simc

5. simcore Threads

When configuring simcore, you choose which threading model to use:

   ./configure --with-threads

Multi-threaded (nPth): Your eventually multi-threaded program can call the simcore API from any thread; simcore threads run simultaneously with your thread(s). Threads started by libraries used by simcore run simultaneously with simcore threads and your thread(s). This is the default mode.

   ./configure --without-threads --without-pth

Single-threaded (nPth): Your eventually multi-threaded program can call the simcore API from a single thread of your choice. That thread will be blocked while any simcore thread is running, but libraries used by simcore will still run simultaneously with simcore threads and with your thread(s), so some responsiveness is provided.

   ./configure --without-threads --with-pth

Fully single-threaded (Pth): Your eventually multi-threaded program can call simcore from a single thread of your choice. That thread will be blocked while any simcore thread is running; libraries used by simcore will block simcore threads AND the thread from which you called simcore. Avoid using this version with a GUI.

The Pth version may use less CPU time than the nPth version, but it has restrictions on the total number of socket descriptors that can be used. If you configure --without-threads but neither --with-pth nor --without-pth, the default is to use Pth if it is available from the system, else use nPth.

Each simcore thread has a single argument (passed to it on creation), which is usually a pointer to a structure. Threads that require interactivity communicate with other threads by using Pth message queues. These are specially implemented by spth.c in a system-dependent way, so that writing to a message queue is non-blocking (not possible with the nPth API). The only other lock (synchronization primitive) used by simcore, apart from the main nPth lock and the logger lock, is the socket lock (one for each simsocket). The socket lock is needed because simcore lacks a sender thread, so multiple threads can send to a single socket at the "same" time, and this needs to be synchronized.

Here's an overview of all simcore threads:

Notes:

• a bold thread name indicates that there may be multiple threads of this type running simultaneously
• a bold struct indicates that a Pth message queue is used with the thread
• audio_close_ is called by sim_audio_call_, sim_audio_hangup_, thread_hangup_, audio_init_, audio_ring_ and audio_reopen_
• audio_init_ is called by sim_audio_call_, sim_list_info_, thread_hangup_, thread_pnp_, audio_close_, audio_ring_ and sim_exit_
• audio_ring_ and audio_reopen_ are called by thread_connect_ and thread_server_
• msg_connect is called by sim_contact_ping_, sim_contact_connect_, sim_audio_call_, sim_audio_hangup_, sim_msg_send_utf_, sim_msg_edit_utf_, sim_msg_remove_, sim_xfer_send_file_, sim_xfer_set_, sim_param_set_, thread_probe_, thread_connect_ and thread_server_.

simcore threads use up to 32K stack space each, but the thread stack limit is set to 64K to provide extra safety. If you modify simcore, avoid writing recursive functions and putting large arrays on the stack.

Note that libraries used by simcore (such as portaudio) also start background threads when necessary.

6. Notable simcore Structures

The most important simcore structure is simtype as it is heavily used internally, and is also used by the simcore public API. simtype defines a dynamically-typed (and usually dynamically-allocated) value, as described by api.html.

You can think of it as a typed pointer with a "length". For the SIMNUMBER type, it contains the value of a number (not a pointer to it), but this should be considered an implementation detail.

Another important structure associated with simtype is simwalker (used as an opaque context when iterating over all values of a SIMTABLE type). struct _simelement is the basic building block of a SIMTABLE.

simtypes can be nested recursively (used as containers of dynamically-typed values). Note that if such a container (SIMTABLE or SIMARRAY) contains a SIMPOINTER, it does NOT "own" the string that the SIMPOINTER points to, and does not free it from memory when destroyed. If it contains any other type, the simtype container owns it and frees it when destroyed. simtypes can be detached from a simtype container in order to remove them from the container AND transfer ownership from the container simtype to the caller at the same time.

The following table lists some more or less important public (simcore internal API) structures:

The next table lists some relatively important structures that are private to each of the listed modules: