This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Accessing shared DSPLink memory from another process.

Other Parts Discussed in Thread: DA8XX

Software versions: DSPLink 1.65 on TimeSys Linux (Linux omapl137_evm 2.6.29-ts-armv5l #1 PREEMPT Mon Oct 19 12:24:38 EDT 2009 armv5tejl GNU/Linux)

I have an application A that sets up DSPLink, creates a pool, allocates a buffer and starts a DSP executable.  I have an application B which is passed the shared memory pool which is passed the memory address of the pool.  When I try to access that memory from app B, it crashes with a segmentation fault.  App A is, of course, fine.
I believe the reason app B seg faults is because it does not have access to the dsp shared region in it's memory map.  However, when I try to add it with mmap (using the method below), I get an error from mmap saying "No such device."  Any hints as to how to access the shared memory region allocated in app A?

 int Application::mapFlagAddress() {
    int result = 0;
    int pa_offset;
    int pageSize = sysconf(_SC_PAGE_SIZE);
    unsigned char *flag_mem; // keep track of this?
    if(flag_fd = open("/dev/mem", O_RDWR) < 0) {
        perror("Application::mapFlagAddress() can not open /dev/mem");
        result = -1;
    }
    // I think this buffer is too big.
    else if((flag_mem = (unsigned char*)malloc(controlCom->flagBufferSize + pageSize)) == NULL) {
        perror("Application::mapFlagAddress() could not allocate memory to map");
        result = -1;
    }
    else {
        printf("flag_mem 0: %x\n", (unsigned int)flag_mem);
        if((unsigned long)flag_mem % pageSize) {
            flag_mem += pageSize - ((unsigned long)flag_mem % pageSize);
        }
        // make sure this is right.
        if(controlCom->flagBufferSize % pageSize) {
            controlCom->flagBufferSize = (controlCom->flagBufferSize / pageSize) + pageSize;
        }
        printf("flag_mem 1: %x\n", flag_mem);
        }
        // make sure this is right.
        if(controlCom->flagBufferSize % pageSize) {
            controlCom->flagBufferSize = (controlCom->flagBufferSize / pageSize) + pageSize;
        }
        printf("flag_mem 1: %x\n", flag_mem);
        printf("flagBufferSize: %x\n", controlCom->flagBufferSize);
        // calculate the memory offset, must be page aligned.
        pa_offset = (int)(controlCom->armFlagAddress) & ~(sysconf(_SC_PAGE_SIZE) -1);
        flag_mem = (unsigned char *)mmap(
            (caddr_t)flag_mem,
            controlCom->flagBufferSize,
            PROT_READ | PROT_WRITE,
            MAP_SHARED,
            flag_fd,
            pa_offset
            );
        if((long)flag_mem < 0) {
            perror("Application::mapFlagAddress() could not map memory");
            result = -1;
        }
        else {
            flag_mem = flag_mem + ((int)controlCom->armFlagAddress - pa_offset);
            flagAddress = (Flags *)flag_mem;
        }
    }
    return result;
}
Thanks,
Chris 
  • There is a DSPLink sample that seemingly demonstrates what you're trying to do.  Take a look at <DSPLINK>/gpp/src/samples/mapregion/mapregion.c, where  you will find this code:
            mapInfo.dspAddr = dsp_addr2 ;
            mapInfo.size    = bufferSize ;
            status = PROC_control (processorId,
                                   PROC_CTRL_CMD_MAP_DSPMEM,
                                   &mapInfo);

    Call the above code from app B.

    Regards,

    - Rob

     

  • Thanks, Rob!  That looks useful.  I've given this a try, but with no luck.  I'm getting DSP_EFAIL back from the all to PROC_control.

    Here's what I've done (briefly).

    App A: 

    PROC_setup(NULL);

    PROC_attach(0, NULL);

    POOL_open(...)

    POOL_alloc()

    POOL_translate() alloc'd address to dsp address

    PROC_load(my dsp image)

     

    App B:

    PROC_setup(NULL)

    PROC_attach(0, NULL)

    PROC_control() with dspAddr set to the dspAddress from App A

     

    Running this, I get 0x80008008 back from the system.  This is DSP_EFAIL.  Looking through the source code for the PROC_control routine, I find it returns an EFAIL if the mmap routine fails....

    any idea what might be happening here?  I can attach the actual code samples if that would be helpful.

  • As you said, mmap() on the DSPLink driver is failing.  It is returning MAP_FAILED, and according to the mmap() man page, when it returns MAP_FAILED it further sets errno to specify the error more precisely.  Can you check errno after the failure?

    It wouldn't hurt to attach your code samples, if you would, I can sanity check you.  I don't have a good idea of why this might fail.

    Regards,

    - Rob

  • I'm getting "Invalid argument" after I call perror() immediately following the PROC_control call.  I've attached dspload.c, which is the program that loads the DSP image and initializes the pool. application.cpp has a routine called setupDSPLink that makes the call to PROC_control.

    Thanks,
    Chris 

    0317.dspload.txt

    3386.application.txt

     

  • OOps.  ignore the line in application.cpp that reads printf("errno: %d\n:", errno);  Forgot to take that out before I sent it to you.  :)

  • Sorry for the delay, I got sidetracked with some priority work...

    I looked at your code and noticed something amiss:
                mapInfo.dspAddr = (Uint32)controlCom->dspFlagAddress;
                mapInfo.size = controlCom->flagBufferSize;
    //            mapInfo.mappedAddr = (Uint32)controlCom->armFlagAddress;
    //           mapInfo.mappedSize = controlCom->flagBufferSize;

                status = PROC_control(0, PROC_CTRL_CMD_MAP_DSPMEM, &mapInfo);
    You're using the mapInfo.dspAddr field to hold your DSP address, but the PROC_CTRL_CMD_MAP_DSPMEM PROC_control() call uses the mapInfo.mappedAddr (which you actually are using in your commented-out statements).  'dspAddr' is actually a member of the ProcMemMapInfo structure, but it's not used in that PROC_control() call.

    Give that change a shot and we'll go from there if you're still having trouble.

    Regards,

    - Rob

  • Hi Rob,

    I set up those lines of code while hacking around... just to see.  I've reverted back to the commented out version (mappedAddr and mappedSize).  I'm now getting this output:

    controlCom->armFlagAddress: 40193200
    controlCom->flagBufferSize: 1024
    Error calling PROC_control: 0x80008010
    PROC control: Interrupted system call

    The mmap inside PROC control appears to be returning an Interrupted System call error.  I'm looking to see what that means but haven't found anything yet.  DSPLINK is returning an odd error code, one I don't see in errbase.h (on quick inspection.)
    Thanks,
    Chris

  • UPDATE: You may want to jump ahead to my next reply before trying anything that I suggest in this already-posted post...

    Christopher Fury said:
    controlCom->armFlagAddress: 40193200
    controlCom->flagBufferSize: 1024

    Why are you using the ARM address?  I saw that commented-out code:
                mapInfo.dspAddr = (Uint32)controlCom->dspFlagAddress;
                mapInfo.size = controlCom->flagBufferSize;
    //            mapInfo.mappedAddr = (Uint32)controlCom->armFlagAddress;
    //           mapInfo.mappedSize = controlCom->flagBufferSize;

    Your previous attempt was with 'dspFlagAddress' but assigning it to the wrong struct element in mapInfo.  Now, assuming you're using the commented-out code above, you're using the right struct member (mappedAddr) but with the 'armFlagAddress', which is already a virtual address.  Please try:
                mapInfo.mappedAddr = (Uint32)controlCom->dspFlagAddress;
                mapInfo.mappedSize = controlCom->flagBufferSize;

    Christopher Fury said:
    Error calling PROC_control: 0x80008010
    I'm looking to see what that means but haven't found anything yet.  DSPLINK is returning an odd error code, one I don't see in errbase.h (on quick inspection.)

    errbase.h is a bit tricky to read, but here's the line:
        #define DSP_ENOTIMPL                (DSP_EBASE + 0x10l)
    The 'l' (lowercase L) obscures the integer display.

    I will try to see what's causing that return from PROC_control() and post back here.

    Regards,

    - Rob


  • Christopher Fury said:
    Error calling PROC_control: 0x80008010

    I'm afraid I may have sent you down the wrong path here in advising you to use PROC_control()...

    I just discovered that before calling the mmap() function, DRV_Invoke() (which is called by PROC_control() for cmd CMD_PROC_CONTROL) performs an ioctl(CMD_PROC_CONTROL) call down to the DSPLink device driver, and DA8XX version (which covers DA830 which is aka OMAPL137) is just a shell function that returns DSP_ENOTIMPL, which is the return code that you're getting from PROC_control().  The perror() call that prints "Interrupted system call" is a red-herring here - the DSPLink error here does not result in an errno-based error, so you most likely have a leftover errno value of EINTR from some previous system call.

    Had the DRV_Invoke() ioctl() not failed, mmap() would be called, but alas.

    I will ask around for a solution to your issue.

    Regards,

    - Rob

     

  • Hi Rob,

    I tried both the dsp and the arm address, because I wasn't entirely sure what was going on here. :)  I'll wait to hear back from you about how to proceed before I try again -- I'm hunting down a DSP side issue right now...

    Thanks for all your help so far!  I really appreciate it.

    Chris

  • Chris,

    I got some advice from a DSPLink expert.  They state that one possible solution would be to modify DSPLink to do what you want, i.e., if the ioctl() in the case CMD_PROC_CONTROL returns DSP_ENOTIMPL then instead of failing the call it could just proceed to do the mmap().

    They also stated, when asked by me if it would work, that App B should be able to open the same POOL as the one on App A, and do a POOL_translateAddr() from App B.  This is allowed because the 2nd App would simply reference the POOL (and perform a new user mapping) opened by the first App instead of creating it.

    If you want to use the POOL buffer with other DSPLink modules, such as MSGQ, then the POOL_open() method must be used (and not the hacked mmap() method.

    Sorry for the loose advice here, I've steered you wrong in a few places and perhaps am steering you down a path to failure here as well, but we'll struggle through this together.  Let me know how it goes.

    Regards,

    - Rob

  • Here's what I tried:

     if(DSP_SUCCEEDED(status)) {

                debug("controlCom->armFlagAddress: %x\n", (unsigned int)controlCom->armFlagAddress);

                debug("controlCom->flagBufferSize: %d\n", controlCom->flagBufferSize);

     

                status = POOL_open(POOL_makePoolId(0, 0), NULL);

     

                if(DSP_SUCCEEDED(status)) {

                    status = POOL_translateAddr(POOL_makePoolId(0, 0),

                            (void **)&flagAddress,

                            AddrType_Usr,

                            controlCom->dspFlagAddress,

                            AddrType_Dsp);

                    if(DSP_FAILED(status)) {

                        perror("POOL_translateAddr");

                        log("Error calling POOL_translate: 0x%x\n", status);

                    }

                }

                else {

                    perror("POOL_open");

                    log("Error calling POOL_open: 0x%x\n", status);

                }

    }

    And I get:
    POOL_open: Interrupted system call
    Error calling POOL_open: 0x8000800b
    Questions:
    1) I'm passing in null parameters into the POOL_open, I assume this is Ok since I'm opening a pool that's already configured.  When I add in the boiler plate code:
    #define NUMBUFFERPOOLS  2
    #define NUMBUFS_POOL0   4
    #define NUMBUFS_POOL1   4
    #define CONTROL_BUFFER_SIZE DSPLINK_ALIGN(sizeof(Uint32) * 4, DSPLINK_BUF_ALIGN)
    and
     Uint32 numBufs[NUMBUFFERPOOLS] = {NUMBUFS_POOL0, NUMBUFS_POOL1};
                Uint32 size[NUMBUFFERPOOLS];
                SMAPOOL_Attrs poolAttrs;
                size[0] = controlCom->flagBufferSize;
                size[1] = CONTROL_BUFFER_SIZE;
                poolAttrs.bufSizes = (Uint32 *) &size;
                poolAttrs.numBuffers = (Uint32 *) &numBufs;
                poolAttrs.numBufPools = NUMBUFFERPOOLS;
                poolAttrs.exactMatchReq = TRUE;
                status = POOL_open(POOL_makePoolId(0, 0), &poolAttrs);
    The application now runs, but the memory has odd values in it and when I make a change, it does not persist to the next run of the second app.   (nothing sees the changes.)
    Am I doing this right?
    Thanks,
    Chris 
    Thanks,
    Chris 

  • Christopher Fury said:
    And I get:
    POOL_open: Interrupted system call
    Error calling POOL_open: 0x8000800b

    The perror() print is probably stale.

    0x8000800b is DSP_EINVALIDARG.  POOL_open() can return DSP_EINVALIDARG if the pool ID is out of range, for which it checks first thing in the API, but clearly POOL_makePoolId(0, 0) is within range.  Deeper down into POOL_open(), DSPLink kernel code can percolate DSP_EINVALIDARG up to the user layer if the kernel POOL state object doesn't know about the pool.  It seems to me that the DSPLink kernel code thinks there are no pools in the system.

    Are you sure you're running the POOL_open(POOL_makePoolId(0, 0), NULL) after running the other App that performs the full POOL_open(..., &poolAttrs), and that the other App hasn't closed the POOL?

    The fact that it sort of works when providing non-NULL params suggests to me that the POOL hasn't already been created by the time you call POOL_open() with NULL params.

    Christopher Fury said:
    1) I'm passing in null parameters into the POOL_open, I assume this is Ok since I'm opening a pool that's already configured.

    Yes, this should work when the NULL params is used *after* the non-NULL params POOL_open() has been called in the other app.

    I will continue to look into this, to confirm that this model (which is supported by the DSPLink design) should actually work, but I wanted to get back to you with some answers before deep-diving into DSPLink code.

    Regards,

    - Rob

     

  • Rob,

    It works!  Here's what I did:

    1) I switched back to POOL_open(POOL_makePoolId(0,0), NULL)

    2) ensured that the program that opens the pool stayed running (put it in a loop)

    I now have shared memory access.  Thank you for your help!

    A follow up question: is it possible to keep a pool open even if the executable that opened it shuts down (without calling the pool close command?)

    Thanks,
    Chris 

  • Christopher Fury said:

    I now have shared memory access.  Thank you for your help!

    You're welcome, glad I could help.

    Christopher Fury said:

    A follow up question: is it possible to keep a pool open even if the executable that opened it shuts down (without calling the pool close command?)

    DSPLink  has an "exit handler", named DSPLINK_atExitHandler() (defined in drv_api.c).  One of the things it does is close all POOLs:
    #if defined (POOL_COMPONENT)
        for (i = 0 ; i < MAX_POOLENTRIES ; i++) {
            POOL_close (i) ;
        }
    #endif /* if defined (POOL_COMPONENT) */
    You could modify this code to not close the pools, although I have no idea what implications there would be.

    Or, since the DSPLink kernel module reference-counts the opens and closes of POOLS, you could ensure that the 1st App that opened the POOL doesn't close it until the 2nd App has opened the POOL, at which point the 1st App can close the POOL and the reference count would simply be decremented w/o closing the POOL.

    Regards,

    - Rob

     

  • Ok.  I'll probably just leave the app running in the background listening for a shutdown signal.  Thanks a bunch!

    Chris