This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

How to find out the memory address for loading symbols for dynamically linked libraries?

Hi,

We have DSP SW running on Keystone II device using dynamic loader as described in http://processors.wiki.ti.com/index.php/C6000_Dynamic_Linking.

DSP loads dynamic libraries through ARM but since the libraries are dynamic their memory location is not that obvious. 

Is there a certain variable in dynamic loader implementation which would tell me the exact address where I could load the debug symbols (.out file) for the dynamic library in CCS?

regards,

Marko

  • Hello Marko,

    In the reference implementation of the dynamic loader, the function DLIF_allocate() is used to get a chunk of DSP memory where each segment of a dynamic library is going to be loaded. On the client side of the dynamic loader (dlw_client.c) DLIF_allocate() calls DLTMM_malloc() to allocate each chunk of target memory (DLTMM indicates the Dynamic Loader Target Memory Manager). DLTMM_malloc() in turn calls trg_alloc() which will find a free target memory packet on the list of TRG_PACKET objects to which trg_mem_head points.

    One alternative is to set up your own map of dynamically loaded modules that gets updated each time DLIF_allocate() or DLIF_free() is called.

    Another possibility is to adapt the dynamic loader's generation of the "DLL View" debug list so that it works with a dynamic loader that is running on the host (the reference implementation requires that the dynamic loader resides and runs on the DSP target and defines a symbol called "_DLModules" in order to generate the DLL View debug list). See dlw_debug.[ch] for details on the DLL View debug list.

    Hope this helps.

    Regards,

    Todd Snider

    C6000 Code Generation Tools Team

    Texas Instruments Incorporated

  • Thanks for the tips. Let me just double check a few details.

    If I want to track down the address where the dll is loaded, should I be looking at TARGET_ADDRESS in DLOAD_MEMORY_SEGMENT? Is that the address I should be using when loading in the .out file with debug symbols? 

    struct DLOAD_MEMORY_SEGMENT
    {
       uint32_t       target_page;          /* requested/returned memory page    */
       TARGET_ADDRESS target_address;       /* requested/returned address        */
       uint32_t       objsz_in_bytes;       /* size of init'd part of segment    */
       uint32_t       memsz_in_bytes;       /* size of memory block for segment  */
    };

    I would like to make sure that I get the symbols to the right address manually before thinking about more complicated solutions such as the one implemented in dlw_debug.

    regards,
    Marko



  • Hi Marko,
    Yes, "target_address" is the address of the memory allocated at dynamic load time to hold whatever is being loaded.
    If you consider the code in dlw_debug.[ch], this might prove useful to you in getting the exact location of each loaded segment. Look for references to "DLL_debug" in dlw.c and dlw_client.c. When DLL_debug is enabled, the dynamic loader keeps a record of what modules are loaded and where they are. While a module is in the process of being loaded, a record for it is created and pushed onto the top of a "context stack" (allowing for dependent modules to be loaded as well). When the load of a particular module is completed, its identity and location are recorded in the debug record for it, that debug record is then popped off the context stack and added to the "mirror_debug_list".
    If you were to maintain the context stack and mirror_debug_list in host (ARM) memory, then you should be able to identify where any given loaded module is located. There is a helper function, DLDBG_dump_mirror_debug_list(), that will write out the content of the mirror_debug_list at any given time.
    Hope this proves helpful.Regards,Todd Snider
  • Hi there,

    One more thingy. I was hoping to have a simple solution that would work for dynamic libraries in the same way as "load symbols" in CCS works for static .out files. Is it even possible to use CCS to load symbols of a dynamic library to a single memory location defined by a code offset in "load symbols" dialog box? Or do I have to  somehow manually parse the .out file segment by segment and load the symbols accordingly?

    Marko

  • Hi Marko,
    If you are truly doing dynamic linking and loading, then the load address of any loaded segment will be determined at load time.
    If you are dealing with a static executable, then its load address is determined during the static link. For such an output file, you can find the actual load address of a given program segment in the program header table.
    I am not aware to what extent CCS supports dynamic linking, if at all.Regards,
    Todd Snider
  • Well, well. My question from the day one has been about using CCS to load symbols for dynamic libraries. And now after two months you are saying that you don't know anything about the topic. I am quite disappointed I must say. If you are not able to help, can you please forward this question to someone who can.

    I can dig out the memory location where segments are loaded but how do I get the debug symbols imported to CCS so that I can actually debug/step through the C code in dynamic library? Disassembler debugging works of course.

    Marko

  • Hi Marko,
    I apologize for the misunderstanding. I had been under the impression that you were using the reference implementation of the dynamic loader to do dynamic linking and loading. That is the body of source code that the C6000_Dynamic_Linking wiki article references as an example of the Bare Metal Dynamic Linking Model.

    Are you running the reference implementation of the dynamic loader on CCS? or are you trying to use CCS to perform dynamic linking and loading? Perhaps this is the crux of my confusion.
    Regards,Todd Snider
  • Thanks for coming back on this. Let me try to give you a bit more information so that we are on the same page.

    We have a DSP base image which is more or less normal, statically linked image which runs on each DSP core. ARM runs linux and DSP base image is loaded to DSP cores using mpm Linux user space application. Once this is done, I can connect CCS to target HW and load symbols for the base image (.out file) and step/debug the code normally.

    The DSP base image has capability of loading and unloading dynamic libraries when needed. These dynamically linked DSP libraries are stored in Linux file system and ARM hosts to file service so that DSP can request libraries when needed. Eventually libraries are loaded and DSP can call any functions which are in the dynamically liked libraries. This all is implemented based on the TI's reference implementation.

    The issue I am having with the dynamically linked libraries is that when I connect CCS to target HW and step from the base image to dynamic loader and then to dynamically linked library, I don't have the debug symbols available. I can step through the disassembly but that is quite painful. If that was a static library I would just use "load symbols" from CCS and load the missing debug symbols to give me access to C source view and variables (just like with the base image). The question is, how do I get the debug symbols (function names, source view, variables names etc.) for dynamically linked libraries? CCS allows loading of symbols (from .out file) to a specific address (code offset) and I was hoping that that would be the solution but apparently it is not that simple. 

    I don't know if the dynamic loader has all the ELF segments in the same order and back to back in a way that the loading of symbols with a give start address would even work.

    regards,

    Marko

  • Placeholder for the missing piece of information seems to already be in wiki page: http://processors.wiki.ti.com/index.php/C6000_Dynamic_Loader#DLL_Debug_Support

    Marko
  • Hi Marko,
    The discussion from my earlier response (Jan 5 @ 1:52pm) is referring to the "DLL Debug Support" in the reference implementation of the dynamic loader (RIDL). This might help you to determine where in target memory a given DLL is loaded. However, I think the essential problem is that CCS is only giving you visibility into the data structures of the RIDL, CCS does not have direct access to the dynamic symbols that get loaded *by* the RIDL. The RIDL does have access to these symbols via the symbol tables attached to a loaded dynamic module (see DLIMP_Dynamic_Module declaration in dload.h of the RIDL sources).
    I speculate (since I have not actually tried to debug dll code via a CCS that is running the RIDL that was used to load the dll) that you might be able to use CCS to get to the loaded dll module's symbol table in RIDL to find dynamically loaded symbols, but this doesn't sound like an effective solution to me. You may have better luck annotating the RIDL to write to stdout the dll's load address and its symbol information as it is loaded by the RIDL. Once you know the location of a given symbol, you should be able to watch that location in CCS while the dll code executes.
    Todd
  • Hi Marko,

    Marko Moberg said:
    My question from the day one has been about using CCS to load symbols for dynamic libraries

    CCS does not have any default support for this. However this can be accomplished by using some custom scripts that can determine which libraries have been loaded by the dynamic loader and load the appropriate debug symbols for debug. I once created such a script long time ago for a customer (with help from Todd). Unfortunately it is no longer supported but I'd like to attach it here because I think it may still be useful in your case too. The script is well documented with the instructions in the comments too. I've attached it here (it is a *.js file that I renamed as a *.txt):

    dllView_elf.txt
    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    /*
    * dllView_elf.js
    *
    * dllView.js port to support ELF. This script can determine which dll has been
    * loaded by the dynamic loader and will automatically load the appropriate
    * symbols needed for debug.
    *
    * Versions Author
    * 0.1 Ki-Soo Lee
    * 0.2 Ki-Soo Lee
    * 0.3 Ki-Soo Lee
    * 0.4 Ki-Soo Lee
    *
    * ==============================================================================
    *
    * Dependencies:
    * 1) CCS v4.2.1 or greater
    * 2) Application built with a compiler version which supports dynamic linking
    * 3) Dynamic loader is being used
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

    Note that the script was written for CCSv4 (hence all the references to it in the comments) but it should still apply to later versions of CCS.

    Thanks

    ki

  • Thanks for the script file. I finally had a chance to try it out and it has been very helpful. I had to add a few debug related files from the reference implementation to our dynamic loader implementation but it seems to be working now (more or less).

    Is it really true, that CCS requires that all the objects files have .out extension? At least that seems to be the case. Our dll's were named differently which made GEL_SymbolAddELFRel function fail. But after renaming the library files as .out files the loading seems to be working.

    I'll run a few final tests and then mark this thread as answered/verified.

    regards,

    Marko

  • Marko Moberg said:
    Is it really true, that CCS requires that all the objects files have .out extension? At least that seems to be the case. Our dll's were named differently which made GEL_SymbolAddELFRel function fail. But after renaming the library files as .out files the loading seems to be working.

    No, the GEL call should be able to handle other extension names. What extension name are you using? Can you describe the failure?

  • I am not in the office right now so I can not get you the exact error message, but it is very much the same what you get when trying to load symbols from a file not ending with .out using commands from CCS menu (Load Symbols). The message tells quite clearly that the file should be .out. We have been using .lib extension but it seems that we need to change them to .out to workaround this issue. I am using CCS 5.5 on Linux.

    I'll write down the exact message and steps reproduce the issue tomorrow.
    Marko
  • I see now. The GEL call will accept any extension except .lib. So you can rename the extension to anything else. .lib. It is not the GEL call itself that has this limitation but somewhere in the debugger itself since .lib is a reserved extension name.

    Thanks

    ki