- 1 Project Overview
- 2 How RPC Works
- 3 GPP Side Architecture
- 4 DSP Side Architecture
- 5 Function Signatures
DSP-RPC-POSIX is a component of the C6Run project which allows you to do DSP->GPP remote procedure calls - that is, you can invoke functions/code residing somewhere on the GPP side directly from the DSP as if you were accessing a local function (there are, of course, certain requirements and restrictions).
"What functions are available on the GPP side, then?" one might ask. The answer is pretty much "everything that the hardware can do" or "everything you can do in your regular operating system" or something along the lines of that. From basic tasks like accessing the file system to more sophisticated things like sending a file over FTP, there's a myriad of possibilities.
Two reasons as to why DSP->GPP RPC is desireable would be: access to otherwise (directly) inaccessible features and being able to reuse existing code. The reason why C6RunApp exists is because it's messy to write and run code for the DSP, especially if all you want to do is try out or experiment with things (prototyping). C6RunApp makes your life easier as a DSP-side developer by offering you easy compilation/running and access to console I/O (which is actually a limited form of RPC); DSP-RPC-POSIX expands this by granting you access to virtually any existing functionality you have on the GPP side.
How RPC Works
Step-by-step RPC Events
Let's start by some definitions:
- DSP-side application - this is what you're assumed to be currently working on, which you compile with the C6Run script and will work on the DSP.
- GPP-side application - sets up the DSP app and starts running it, and then "answers" the RPC requests. C6Run actually generates this for you, so you don't have to worry about anything here.
- RPC target - a function which resides somewhere in the GPP side (could be a library, a shared library, your own code, etc.)
- DSP-side stub - a little wrapper function which looks identical to the RPC target. it causes the RPC target to be executed with the parameters you passed to it, and returns you the same value it returns. this is what you actually call from the DSP-side. can be produced by the c6runapp-rpcgen tool, or written manually.
- GPP-side stub - another little wrapper function on the GPP side, this is what the GPP side application actually calls. this function "knows" how to call the RPC target itself, so it executes that call and gets the result. can be produced by the c6runapp-rpcgen tool, or written manually.
The run of events that occur when you want to do a remote procedure call are as follows:
- From inside the DSP-side application, the DSP-side stub is called (which looks identical to the RPC target)
- The DSP-side stub is executed. It initializes the RPC request, and copies all the parameters into the request package (called "marshalling"), and signals for the RPC to be performed.
- The request package is sent to the GPP-side application using the RPC transport.
- The GPP-side application receives the package, unpacks it and extracts the parameters into a buffer (called "unmarshalling"). Some extra processing such as address translation for buffer/pointer parameters may be carried out at this step.
- The GPP-side application locates the relevant GPP-side stub and executes it.
- The GPP-side stub executes the RPC target, using the provided parameters, then stores the return value into another buffer. Some extra processing regarding structures or non-shared buffer return types may be carried out at this step.
- The GPP-side application sends back the result to the DSP-side.
- The DSP-side stub receives the result in the buffer, extracts and returns it to the user code.
Structure of the RPC Package
The buffer carrying a RPC request is structured as follows:
4 NameLen 4 SignatureLen ... 1 +----------+--------+----------------+-------------+----------+---+ | NameLen | Name | SignatureLen | Signature | Params | 0 | +----------+--------+----------------+-------------+----------+---+
- NameLen: length of the function name
- Name: function name of the GPP-side stub to be executed (observe: NOT the name of the RPC target)
- SignatureLen: length of the function signature
- Signature: function signature describing how the parameters section will be unpacked
- Params: the function parameters, packed without any size promotions or alignment
- 0: the null-terminating zero signalling the end of the package
GPP Side Architecture
Relevant source code files: build/gpp_libs/rpc_server.c build/gpp_libs/rpc_server.h build/gpp_libs/cio_ipc.c rpc/gpp/*.c
DSP-RPC-POSIX's GPP-side is "heavier" compared to its DSP-side and almost completely integrated into the C6Run GPP-side library. Aside from C6Run's regular GPP-side duties such as setting up the DSP and serving C I/O requests, it is also responsible for these tasks:
- extracting/cleaning up the GPP stubs library
- receiving and responding to RPC requests
- unmarshalling received packages
- postprocessing stubs' returned data
- locating and executing stubs
- managing RPC memory
The GPP Stubs Library
All GPP stubs located inside the rpc/gpp directory are compiled into a dynamic link library (librpcstubs.so), which allows the usage of dlfcn.h functions dlsym to dynamically locate them by their names. This dynamic link library is rebuilt, converted into a C header file and included in the compilation of the final executable. Upon launch, the library is temporarily extracted into the same directory as the executable, used for locating and executing the stubs, then removed upon termination.
Receiving and Responding to RPC Requests
The important task of servicing RPC requests - that is, carrying out the recieve-unmarshal-locate-execute-return steps, is currently done inside the C I/O service routines (since the RPC transport is carried out via the C I/O transport), located inside build/gpp_libs/cio_ipc.c.
Address Translations and RPC Memory
The GPP side server application is responsible for managing the shared memory regions used in RPC. Central to its workings is the CMEM kernel module which allocates contiguous memory regions and provides both physical and virtual addresses for accessing these. Since the CMEM module offers only physical-from-virtual address translation with base addresses, DSP-RPC-POSIX maintains an internal list of allocated buffers and their sizes to facilitate bidirectional address translation for any memory region lying within the allocated areas (ie, not just the allocated base addresses but incremented pointers referring to somewhere within that allocated block are also translateable). The list is kept as a doubly-linked list, which grows with rpc_malloc calls and shrinks with rpc_free calls. Using rpc_translate_address (which is also exposed through RPC on the DSP-side), the list can be searched to perform virtual->physical or physical->virtual translations. If any address translation goes unsuccessful (no corresponding allocation entries are found) the input address is returned unchanged, which can be used to detect if something has gone wrong.
DSP Side Architecture
Relevant source code files: rpc/core/dsp_core.c rpc/core/dsp_stubs_base.h rpc/dsp/*.c
DSP-RPC-POSIX's DSP-side is very small and rather simple - in fact, none of it currently resides in the C6Run DSP-side libraries but are compiled alongside the user sources every time (thus, modifications to the DSP-side code never need a re-build of the C6Run libraries - just the re-execution of c6runapp-cc script). This is mainly because there isn't all that much to do on the DSP-side: we have a buffer of a certain size (see RPC_BUFSZ in dsp_stubs_base.h) into which every DSP-side stub is responsible for copying its function name, function signature and parameters ("marshaling"), which is then sent to the GPP side using the transport, then the reply obtained and passed back to the DSP stub.
Aside from the information transmitted in the RPC request, there is one more piece of information given to the GPP side which is vital to the servicing of the RPC call - the message identifier, which describes the nature of the RPC message and defined as follows:
- RPC_MSG_REQUEST generic RPC function call request
- RPC_MSG_RESPONSE sent by the GPP side as a reply to every RPC request, both generic and specialized
- RPC_MSG_MALLOC specialized RPC function call request, for memory allocation
- RPC_MSG_FREE specialized RPC function call request, for memory dellocation
- RPC_MSG_TRANSLATE specialized RPC function call request, for address translation
The reason why specialized call requests exist is because these particular functions should not be called from inside the GPP stubs library, but be handled directly inside the RPC server. Since all regular stubs would automatically use RPC_MSG_REQUEST, the specialized functions are defined manually in dsp_core.c - they also don't obey the structural conventions defining the RPC packages (no function name, signature or null terminator is given).
Observe that these identifiers are NOT used as the MSGQ MSG identifier - since the RPC transport is carried out via C6Run's existing C I/O transport, those are always CIO_TRANSFER. These RPC identifiers are carried on the command byte during writemsg for requests and the first byte of parm for responses.
Currently, the RPC transport is completely carried out via C6Run's existing C I/O transport system - that is, the functions writemsg and readmsg.
The definitions of writemsg and readmsg, and the usage conventions for parameters are as follows:
void writemsg(unsigned char command, const unsigned char *parm, const char *data,unsigned int length)
- command - the RPC message identifier
- parm - unused, any 8-element char array
- data - the RPC buffer
- length - length of the RPC buffer
void readmsg(register unsigned char *parm, register char *data)
- parm - char array whose first element should contain RPC_MSG_RESPONSE
- data - buffer that contains the RPC response
The function signature is a string of characters describing the data type of a function's return value and parameters. The reason why this data is needed is threefold:
- the unmarshaller needs to know how many bytes each parameter takes while extracting them from the buffer
- GPP and DSP address spaces aren't the same, and in order to know when to perform address translation the unmarshaller needs to know which parameters are pointers (should be translated) and which parameters are regular values (should be left untouched)
- special treatment (such as copying into a shared buffer) may be needed for some functions' return values
The signature is composed of ASCII characters, starting with the character representing the return type and continuing with characters representing each parameter in order. Its length always has to be nonzero (the return type is always needed).
Table of Function Signature Characters
|Signature||Data Type||Size (bytes)||Allowed Positions||Description|
|v||void||0||all||void data - can be omitted for parameters but always needs to be explicitly specified for return types!|
|c||char||1||all||char data, or any other 1-byte data|
|s||short int||2||all||short int data, or any other 2-byte data|
|i||int||4||all||int data, or any other 4-byte data|
|f||float||4||all||float data, i can also be used instead, provided for convenience|
|d||double||8||all||double data, or any other 8-byte data|
|@||pointer||4||all||pointer that needs address translation - see "Pointers and Shared Memory" section for details|
|a||pointer||4||all||pointer which won't be dereferenced and doesn't need address translation - see "Pointers and Shared Memory" section for details|
|$||pointer||4||return||manually copy a number of bytes from the returned pointer into a shared buffer and return the shared buffer - see "Returning Non-Shared Buffers" section|
|#||struct||*||return||manually copy a number of bytes from the returned pointer into the result buffer and return the result normally - see "Returning structures" section|
|>||*||*||param||variadic parameter block: previous parameter is assumed to be containing a printf-style format string, all variadic pointer parameters are address translated using the info from the format string|
|<||*||*||param||variadic parameter block: previous parameter is assumed to be containing a scanf-style format string, all variadic parameters are pointers and are subjected to address translation using the info from the format string (i.e, how many variadic parameters there are)|
Observe that indirect pointers (double/triple pointers such as char**, void**) are not fully supported - the contained direct pointers won't be translated, and neither will be pointers hidden inside struct's
There are two important issues that need to be kept in mind while working with buffers/pointers in RPC:
- The GPP and the DSP don't use the same address space: the GPP works with virtual addresses, while the DSP works with physical addresses
- Due to memory protection issues, a buffer which will be accessed by both the DSP and the GPP must be allocated from a shared memory area (that is, CMEM)
To save you from the troubles of having to work manually with allocating CMEM buffers and translating addresses back and forth, DSP-RPC-POSIX offers you easy allocation of shared buffers via rpc_malloc and automatic translation of memory addresses using the @ character in function signatures. Thus, any rpc_malloc'd buffer you pass as an '@' parameter will be automatically converted to its virtual equivalent which can be used by and GPP side function, and any '@' return type lying inside rpc_malloc'd areas will be translated to its physical equivalent which is accessible by the DSP. It is important to keep in mind that address translations (which are absolutely necessary for accessing pointers from both sides) are performed automatically only in case if the parameter is specified as @ in the function signature. This means that double/triple pointers, pointers inside structs or pointers hidden inside buffers won't be automatically translated.
For the developer's convenience while working with read-only string parameters, DSP-RPC-POSIX has the ability to copy a fixed number of bytes from the DSP memory space (stack or heap) into a GPP-side buffer using PROC_read. This means that as long as the sent buffer is relatively small (defined as RPC_PROCREAD_MAXPARAM in build/gpp_libs/rpc_server.h) and is expected to be read only (ie, not modified on the GPP side) it's safe to pass them from the DSP stack or heap. So rpc_puts("hello world!") or rpc_printf("String: %s \n", "test") is indeed possible.
In case you won't be accessing the contents of a pointer from the DSP-side, it is safe to use the 'a' signature character instead of '@'. In this case the system won't perform any address translation at all, pointers will be passed back and forth as regular values. For example, FILE pointers used by fopen/fread/etc. calls are never meant to be dereferenced at all, but merely passed as arguments to other members of the same function family, and thus are good candidates for being 'a' parameters.
In case the GPP-side function is going to return a pointer to a GPP-side allocated memory buffer (ie, anything that wasn't allocated by rpc_malloc), the function signature return type should be defined as '$' which has this special meaning: the GPP stub needs to use the RPC_LOAD_RESULT_BUFLEN(result_buffer, length) macro to set the size of the returned buffer. The RPC system will copy this many bytes into a shared memory region from the returned buffer, and return the physical address of the shared memory region instead. Observe that this shared buffer is only synchronized once (upon the completion of the GPP side call) and its later contents will not be kept in sync with the nonshared GPP buffer.
In case the GPP-side function is going to return a structure (the structure itself and not a pointer to it), the function signature return type will be defined as '#' which has this special meaning: the GPP stub needs to allocate sufficient memory with malloc(), place the returned structure into this buffer, use the RPC_LOAD_RESULT_BUFLEN macro to set the length of the returned structure, and then return this buffer without freeing the memory. The GPP side server will copy the specified number of bytes into the return buffer and free the allocated memory. Observe that the size of the RPC return buffer is limited (defined as RPC_RESPSZ)