This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Could not find remoteproc0 trace log

Hi ,

I am working in K2HK evm board and also we have developed  own customize board with 66AK2H processor board.

I am trying to load dsp elf through arm using mpmcl utility,

I am able to load the image in dsp and able to execute it. But i could'nt able to see any log of it in directory,

/sys/kernel/debug/remoteproc/remoteproc0/trace0

and i could'nt find remoteproc0 directory in /sys/kernel/debug/remoteproc/ path.

Am i missing any thing,

Please help to sort out this issue,

  • Vidya,

    Do you see this behavior with TI provided example(mpmsrv_keystone2_example)? I have attached pre-built binary of the example. 

    https://e2e.ti.com/cfs-file/__key/communityserver-discussions-components-files/791/mpmsrv_5F00_keystone2_5F00_example.out

    Thank you.

  • Hi Raja,

    Thanks for your input,

    I have tried with the default pre built binaries which is in mcsdk folder and also with your binary but still i could not get debug prints.

       As per the link ,

    Command "cat /sys/kernel/debug/remoteproc/remoteproc0/trace0" show the print message,

    But i could not find remoteproc0 directory in remoteproc folder at all.

    Should i need to create remoteproc0 directory in /sys/kernel/debug/remoteproc/ path or it will created by any script.

    Please clarify,




  • Thank you for confirming. Are you working on latest Processor SDK or MCSDK?
  • Yes,
    I am using k2hk-evm-sdk-bin-02.00.01 and tisdk-server-rootfs-image-k2hk-evm.tar.gz filesystem
  • Any inputs Raja ??
  • I think this feature is broken in Processor SDK release. If you have time, please check this works on latest MCSDK(3.1.4) release binaries (Linux and file system image).

    Give us some time to confirm. Thank you.
  • HI,

    I have been working in this sdk for quite long time and reverting whole our changes to newer MCSDK(3.1.4 ) is difficult and also takes more time.

    I had discussion with TItus on topic "where to get Ramdisk filesystem source for ti K2HK" and she referred this SDK (tisdk-server-rootfs-image-k2hk-evm.tar.gz)

    and now you are referring to newer release. 


    Please add support to this SDK and confirm its working at your end.

    So i can use it.


  • Command "cat /sys/kernel/debug/remoteproc/remoteproc0/trace0" show the print message,

    But i could not find remoteproc0 directory in remoteproc folder at all.

    Should i need to create remoteproc0 directory in /sys/kernel/debug/remoteproc/ path or it will created by any script.

    Please clarify,

    You no need to create remoteproc folder, it will create.

    Have you mounted the debugfs filesystem ??
    mount -t debugfs none /sys/kernel/debug

    processors.wiki.ti.com/.../MCSDK_UG_Chapter_Developing_System_Mgmt
  • I have tried this before posting,

    mount -t debugfs none /sys/kernel/debug

    Followed up the link processors.wiki.ti.com/.../MCSDK_UG_Chapter_Developing_System_Mgmt

    Why remotproc0 is not created.
    How it will be created
  • For me "remoteproc0" is created in NFS mode.
    How did you boot the board ?
    NFS or UBIFS ??
    In processor SDK, remoteproc drivers used as module driver, so you need to do build as module & modules_install for the NFS filesystem.

    root@k2hk-evm:/#
    root@k2hk-evm:/# ls /sys/kernel/debug/remoteproc/
    remoteproc0 remoteproc2 remoteproc4 remoteproc6
    remoteproc1 remoteproc3 remoteproc5 remoteproc7
    root@k2hk-evm:/#
    root@k2hk-evm:/#
  • Hi Titus,

    I am porting filesystem in NFS only, I am using SDK (k2hk-evm-sdk-bin-02.00.01) and filesystem tisdk-server-rootfs-image-k2hk-evm.tar.gz.

    Are you using same filesystem ?

    In linux-4.1.13,i have selected "Keystone Remoteproc support " as modules and did module_install to filesystem path.

    Do i need to selected STE-Modem remoteproc support too ?. I dint select this option.

    I could not see remoteproc0 and others listing under remoteproc directory.

    Attached log,

    [    1.968384] usbhid: USB HID core driver
    [    1.973842] nand: device found, Manufacturer ID: 0x2c, Chip ID: 0xac
    [    1.980270] nand: Micron MT29F4G08ABBDAHC
    [    1.984314] nand: 512 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
    [    1.992393] Bad block table found at page 262080, version 0x01
    [    1.999205] Bad block table found at page 262016, version 0x01
    [    2.005567] 3 ofpart partitions found on MTD device 30000000.nand
    [    2.011763] Creating 3 MTD partitions on "30000000.nand":
    [    2.017227] 0x000000000000-0x000000100000 : "u-boot"
    [    2.023816] 0x000000100000-0x000000180000 : "params"
    [    2.030353] 0x000000180000-0x000020000000 : "ubifs"
    [    2.037333] davinci_nand 30000000.nand: controller rev. 2.5
    [    2.043601] platform oprofile-perf.0: set dma_pfn_offset00780000
    [    2.050139] oprofile: using timer interrupt.
    [    2.054506] Netfilter messages via NETLINK v0.30.
    [    2.059295] nf_conntrack version 0.5.0 (7992 buckets, 31968 max)
    [    2.066172] ctnetlink v0.93: registering with nfnetlink.
    [    2.072174] ipip: IPv4 over IPv4 tunneling driver
    [    2.077901] gre: GRE over IPv4 demultiplexor driver
    [    2.083107] ip_tables: (C) 2000-2006 Netfilter Core Team
    [    2.088632] ipt_CLUSTERIP: ClusterIP Version 0.8 loaded successfully
    [    2.095084] arp_tables: (C) 2002 David S. Miller
    [    2.099849] Initializing XFRM netlink socket
    [    2.104188] NET: Registered protocol family 17
    [    2.108723] NET: Registered protocol family 15
    [    2.113215] 8021q: 802.1Q VLAN Support v1.8
    [    2.117603] Registering SWP/SWPB emulation handler
    [    2.139044] 8021q: adding VLAN 0 to HW filter on device eth0
    [    2.156950] IP-Config: Complete:
    [    2.160210]      device=eth0, hwaddr=d4:f5:13:aa:d8:ba, ipaddr=172.17.1.57, mask=255.255.0.0, gw=172.17.0.3
    [    2.170054]      host=172.17.1.57, domain=, nis-domain=(none)
    [    2.175846]      bootserver=172.17.1.146, rootserver=172.17.1.146, rootpath=
    [    2.282033] VFS: Mounted root (nfs filesystem) on device 0:15.
    [    2.288799] devtmpfs: mounted
    [    2.292100] Freeing unused kernel memory: 316K (c06d5000 - c0724000)
    INIT: version 2.88 booting
    [    3.458108] NET: Registered protocol family 41
    Starting random number generator daemonread error
    
    read error
    
    .
    [    3.786439] random: dd urandom read with 105 bits of entropy available
    Sat Jan  9 00:07:57 UTC 2016
    INIT: Entering runlevel: 5
    NOT configuring network interfaces: / is an NFS mount
    [    4.738856] random: nonblocking pool is initialized
    udhcpc (v1.23.1) started
    Sending discover...
    Sending select for 172.17.2.177...
    Lease of 172.17.2.177 obtained, lease time 7200
    /etc/udhcpc.d/50default: Adding DNS 172.17.0.1
    /etc/udhcpc.d/50default: Adding DNS 8.8.8.8
    /etc/udhcpc.d/50default: Adding DNS 8.8.4.4
    /etc/udhcpc.d/50default: Adding DNS 202.251.202.3
    Starting system message bus: dbus.
    Starting Dropbear SSH server: [    7.091725] NET: Registered protocol family 10
    dropbear.
    Starting mpmsrv daemon.
    platform: k2hk-evm
    Starting softhsm daemon.
    Starting telnet daemon.
    Starting tiipclad daemon
    GateMP is not supported for this device
    
    Opened log file: lad.txt
    
    numProcessors = 9 id = 0 baseId = 0
    
    Spawned daemon: /usr/bin/lad_tci6638
    
    .
    Starting rpcbind daemon...done.
    starting statd: done
    Starting thttpd.
    Starting Lighttpd Web Server: lighttpd.
    2016-01-09 00:08:00: (/home/gtbldadm/processor-sdk-linux-fido-build/build-CORTEX_1/arago-tmp-external-linaro-toolchain/work/cortexa15hf-vfp-neon-linux-gnueabi/lighttpd/1.4.35-r1/lighttpd-1.4.35/src/log.c.164) server started
    * starting FTP Server: vsftpd... done.
    
    
    #>>>>> LCD 12
    IP Address:
    172.17.1.57
    Enabling thermal zones...
    /etc/rc5.d/S98thermal-zone-init: line 7: /sys/class/thermal/thermal_zone*/mode: No such file or directory
    ***************************************************************
    ***************************************************************
    NOTICE: This file system contains the following GPLv3 packages:
            autoconf
            binutils-dev
            binutils
            bison-dev
            bison
            cpp-symlinks
            cpp
            g++-symlinks
            g++
            gawk-dev
            gawk
            gcc-symlinks
            gcc
            gdb
            gdbc6x
            gdbserver
            libgmp10
            libmpc3
            libmpfr4
            make
    
    If you do not wish to distribute GPLv3 components please remove
    the above packages prior to distribution.  This can be done using
    the opkg remove command.  i.e.:
        opkg remove <package>
    Where <package> is the name printed in the list above
    
    NOTE: If the package is a dependency of another package you
          will be notified of the dependent packages.  You should
          use the --force-removal-of-dependent-packages option to
          also remove the dependent packages as well
    ***************************************************************
    ***************************************************************
    
     _____                    _____           _         _
    |  _  |___ ___ ___ ___   |  _  |___ ___  |_|___ ___| |_
    |     |  _| .'| . | . |  |   __|  _| . | | | -_|  _|  _|
    |__|__|_| |__,|_  |___|  |__|  |_| |___|_| |___|___|_|
                  |___|                    |___|
    
    Arago Project http://arago-project.org k2hk-evm /dev/ttyS0
    
    Arago 2015.12 k2hk-evm /dev/ttyS0
    
    k2hk-evm login: root
    root@k2hk-evm:~# cd /
    root@k2hk-evm:/# lsmod
    Module                  Size  Used by
    ipv6                  338366  30
    rpmsg_proto             6823  0
    virtio_rpmsg_bus       11766  1 rpmsg_proto
    remoteproc             25480  1 rpmsg_proto
    virtio                  6460  2 remoteproc,virtio_rpmsg_bus
    virtio_ring            12987  2 remoteproc,virtio_rpmsg_bus
    root@k2hk-evm:/#
    root@k2hk-evm:/#
    root@k2hk-evm:/# mpmcl ping dsp0
    ping succeeded
    root@k2hk-evm:/# mpmcl status dsp0
    dsp0 is in idle state
    root@k2hk-evm:/# mpmcl load dsp0 m
    media/ mnt/
    root@k2hk-evm:/# mpmcl load dsp0 mpmsrv_keystone2_example.out
    load failed (error: -104)
    root@k2hk-evm:/# mpmcl load dsp0 mpmsrv_keystone2_example.out
    load succeeded
    root@k2hk-evm:/# mpmcl run dsp0
    run succeeded
    root@k2hk-evm:/# ls -l /sys/kernel/debug/remoteproc/
    root@k2hk-evm:/#
    root@k2hk-evm:/#
    root@k2hk-evm:/#
    root@k2hk-evm:/#
    

  • How did you compile the kernel ?
  • make uImage ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- LOADADDR=0x80008000

    Attached .config of my kernel kernel-4.1.13config.txt

  • Titus,

    I have suspect in mpm version in the tisdk filesystem

    Is mpmcl working at your end ??

    I followed the below sequence, and found dsp0 to be in error state and reset is getting failed,

    root@k2hk-evm:/# mpmcl status dsp0
    dsp0 is in idle state
    
    root@k2hk-evm:/# mpmcl load dsp0 mpmsrv_keystone2_example_orginal.out
    load failed (error: -104)
    
    root@k2hk-evm:/# mpmcl status dsp0
    dsp0 is in error state
    
    root@k2hk-evm:/# mpmcl reset dsp0
    reset failed (error: -105)
    
    root@k2hk-evm:/# mpmcl load dsp0 mpmsrv_keystone2_example_orginal.out
    load succeeded
    
    root@k2hk-evm:/# mpmcl status dsp0
    dsp0 is in error state
    
    root@k2hk-evm:/# mpmcl reset dsp0
    reset failed (error: -105)
    
    
    load succeeded
    

  • Hi Vidya,
    You have to use the following commands to build the kernel.

    make distclean ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf-
    make tisdk_k2hk-evm_defconfig ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf-
    make zImage ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf-
    make modules ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf-
    make modules_install INSTALL_MOD_PATH=/usr/local/k2hk_fs_psdk ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf-

    After that I have tried to mount NFS on K2HK board and able to find the remoteproc directory.

    PS:
    You don't need to change any manual config changes for this.
  • While booting it hangs,

    [   89.012361] keystone-rproc 10800000.dsp0: assigned reserved memory node dsp_common_cma_pool
    [   89.020908]  remoteproc0: 10800000.dsp0 is available
    [   89.025917]  remoteproc0: Note: remoteproc is still under development and considered experimental.
    [   89.034986]  remoteproc0: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
    [   89.048915] keystone-rproc 11800000.dsp1: assigned reserved memory node dsp_common_cma_pool
    [   89.057479]  remoteproc1: 11800000.dsp1 is available
    [   89.062486]  remoteproc1: Note: remoteproc is still under development and considered experimental.
    [   89.071600]  remoteproc1: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
    [   89.085402] keystone-rproc 12800000.dsp2: assigned reserved memory node dsp_common_cma_pool
    [   89.093970]  remoteproc2: 12800000.dsp2 is available
    [   89.099034]  remoteproc2: Note: remoteproc is still under development and considered experimental.
    [   89.108130]  remoteproc2: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
    [   89.121919] keystone-rproc 13800000.dsp3: assigned reserved memory node dsp_common_cma_pool
    [   89.130490]  remoteproc3: 13800000.dsp3 is available
    [   89.135499]  remoteproc3: Note: remoteproc is still under development and considered experimental.
    [   89.144617]  remoteproc3: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
    [   89.158468] keystone-rproc 14800000.dsp4: assigned reserved memory node dsp_common_cma_pool
    [   89.167033]  remoteproc4: 14800000.dsp4 is available
    [   89.172040]  remoteproc4: Note: remoteproc is still under development and considered experimental.
    [   89.181134]  remoteproc4: THE BINAR

  • Can you please attach the complete boot up log ?
  • =~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2016.05.19 14:50:29 =~=~=~=~=~=~=~=~=~=~=~=
    
    
    U-Boot 2015.07-00080-g68db1b0-dirty (May 12 2016 - 11:50:43 +0530)
    
    CPU: 66AK2Hx SR2.0
    I2C:   ready
    DRAM:  DDR3 PLL (PLL2) Setup ... 
    DDR3 PLL Setup complete, DDR3A clock now running at 400MHz.
    DDR3A initialization complete 
    1 GiB
    NAND:  512 MiB
    Net:   K2HK_EMAC
    Hit any key to stop autoboot:  1  0 
    U-Boot# 
    U-Boot# setenv run_kern 'bootz ${loadaddr} ${rd_spec} ${fdtaddr}'
    U-Boot# setenv args_net 'setenv bootargs ${bootargs} rootfstype=nfs root=/dev/nfs rw nfsroot=${serverip}:${nfs_root},${nfs_options} ip=172.17.1.57:172.17.1.146:172.17.0.3:255.255.0.0::eth0'
    U-Boot# setenv tftp_root;setenv boot net
    U-Boot# 
    U-Boot# setenv nfs_root '/home/armdsp/k2hk-ti/board-support/k2hk-evm-sdk-bin-02.00.01/board-support-bin/filesystem/tifs'
    U-Boot# setenv name_mon 'skern-k2hk.bin'
    U-Boot# setenv name_fdt 'k2hk-evm-4.1.dtb'
    U-Boot# setenv name_kern 'zImage-4.1.13'
    U-Boot# ping 172.17.1.146
    Using K2HK_EMAC device
    host 172.17.1.146 is alive
    U-Boot# boot
    BOOTP broadcast 1
    BOOTP broadcast 2
    BOOTP broadcast 3
    DHCP client bound to address 172.17.2.2 (1015 ms)
    Using K2HK_EMAC device
    TFTP from server 172.17.1.146; our IP address is 172.17.2.2
    Filename '/k2-fw-initrd.cpio.gz'.
    Load address: 0x88080000
    Loading: *
    TFTP error: 'File not found' (1)
    Not retrying...
    BOOTP broadcast 1
    DHCP client bound to address 172.17.2.2 (2 ms)
    Using K2HK_EMAC device
    TFTP from server 172.17.1.146; our IP address is 172.17.2.2
    Filename '/k2hk-evm-4.1.dtb'.
    Load address: 0x88000000
    Loading: *###
     970.7 KiB/s
    done
    Bytes transferred = 39762 (9b52 hex)
    BOOTP broadcast 1
    DHCP client bound to address 172.17.2.2 (3 ms)
    Using K2HK_EMAC device
    TFTP from server 172.17.1.146; our IP address is 172.17.2.2
    Filename '/skern-k2hk.bin'.
    Load address: 0xc5f0000
    Loading: *####
     1000 KiB/s
    done
    Bytes transferred = 45056 (b000 hex)
    BOOTP broadcast 1
    DHCP client bound to address 172.17.2.2 (3 ms)
    Using K2HK_EMAC device
    TFTP from server 172.17.1.146; our IP address is 172.17.2.2
    Filename '/zImage-4.1.13'.
    Load address: 0x82000000
    Loading: *#################################################################
     #################################################################
     #################################################################
     ##########################################################
     1 MiB/s
    done
    Bytes transferred = 3710600 (389e88 hex)
    K2_BM_15.07-39-g035329c SoC:k2hk built:17:53:33, Jan  8 2016
    
    ## installed monitor, freq [102400000], status 0
    Kernel image @ 0x82000000 [ 0x000000 - 0x389e88 ]
    ## Flattened Device Tree blob at 88000000
       Booting using the fdt blob at 0x88000000
       Loading Ramdisk to 90000000, end 90000000 ... OK
       Loading Device Tree to 8fff3000, end 8ffffb51 ... OK
    
    Starting kernel ...
    
    [    0.000000] Booting Linux on physical CPU 0x0
    [    0.000000] Initializing cgroup subsys cpu
    [    0.000000] Initializing cgroup subsys cpuacct
    [    0.000000] Linux version 4.1.13-g8dc6617 (root@localhost.localdomain) (gcc version 4.7.3 20130226 (prerelease) (crosstool-NG linaro-1.13.1-4.7-2013.03-20130313 - Linaro GCC 2013.03) ) #30 SMP PREEMPT Thu May 19 13:48:17 IST 2016
    [    0.000000] CPU: ARMv7 Processor [412fc0f4] revision 4 (ARMv7), cr=30c5387d
    [    0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
    [    0.000000] Machine model: Texas Instruments Keystone 2 Kepler/Hawking EVM
    [    0.000000] Switching physical address space to 0x800000000
    [    0.000000] Reserved memory: created CMA memory pool at 0x000000081f800000, size 8 MiB
    [    0.000000] Reserved memory: initialized node dsp_common_cma_pool, compatible id shared-dma-pool
    [    0.000000] Reserved memory: created DMA memory pool at 0x0000000820000000, size 512 MiB
    [    0.000000] Reserved memory: initialized node dsp_reserved_mpm_area, compatible id shared-dma-pool
    [    0.000000] cma: Reserved 16 MiB at 0x000000081e800000
    [    0.000000] Forcing write-allocate cache policy for SMP
    [    0.000000] Memory policy: Data cache writealloc
    [    0.000000] PERCPU: Embedded 11 pages/cpu @de3a0000 s13120 r8192 d23744 u45056
    [    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 130048
    [    0.000000] Kernel command line: console=ttyS0,115200n8 rootwait=1 rootfstype=nfs root=/dev/nfs rw nfsroot=172.17.1.146:/home/armdsp/k2hk-ti/board-support/k2hk-evm-sdk-bin-02.00.01/board-support-bin/filesystem/tifs,v3,tcp,rsize=4096,wsize=4096 ip=172.17.1.57:172.17.1.146:172.17.0.3:255.255.0.0::eth0
    [    0.000000] log_buf_len individual max cpu contribution: 4096 bytes
    [    0.000000] log_buf_len total cpu_extra contributions: 12288 bytes
    [    0.000000] log_buf_len min size: 16384 bytes
    [    0.000000] log_buf_len: 32768 bytes
    [    0.000000] early log buf free: 14052(85%)
    [    0.000000] PID hash table entries: 2048 (order: 1, 8192 bytes)
    [    0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
    [    0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
    [    0.000000] Memory: 487400K/524288K available (5003K kernel code, 192K rwdata, 1728K rodata, 248K init, 162K bss, 12312K reserved, 24576K cma-reserved, 0K highmem)
    [    0.000000] Virtual kernel memory layout:
    [    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    [    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
    [    0.000000]     vmalloc : 0xe0800000 - 0xff000000   ( 488 MB)
    [    0.000000]     lowmem  : 0xc0000000 - 0xe0000000   ( 512 MB)
    [    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
    [    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
    [    0.000000]       .text : 0xc0008000 - 0xc069b084   (6733 kB)
    [    0.000000]       .init : 0xc069c000 - 0xc06da000   ( 248 kB)
    [    0.000000]       .data : 0xc06da000 - 0xc070a288   ( 193 kB)
    [    0.000000]        .bss : 0xc070d000 - 0xc0735bcc   ( 163 kB)
    [    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
    [    0.000000] Preemptible hierarchical RCU implementation.
    [    0.000000] Additional per-CPU info printed with stalls.
    [    0.000000] NR_IRQS:16 nr_irqs:16 16
    [    0.000000] Architected cp15 timer(s) running at 102.40MHz (virt).
    [    0.000000] clocksource arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x179dd7f667, max_idle_ns: 440795202122 ns
    [    0.000008] sched_clock: 56 bits at 102MHz, resolution 9ns, wraps every 4398046511103ns
    [    0.000025] Switching to timer-based delay loop, resolution 9ns
    [    0.000349] keystone timer clock @102400000 Hz
    [    0.000710] Console: colour dummy device 80x30
    [    0.000737] Calibrating delay loop (skipped), value calculated using timer frequency.. 204.80 BogoMIPS (lpj=1024000)
    [    0.000758] pid_max: default: 4096 minimum: 301
    [    0.000882] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
    [    0.000899] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
    [    0.001677] Initializing cgroup subsys blkio
    [    0.001700] Initializing cgroup subsys devices
    [    0.001722] Initializing cgroup subsys freezer
    [    0.001759] CPU: Testing write buffer coherency: ok
    [    0.002081] /cpus/cpu@0 missing clock-frequency property
    [    0.002128] /cpus/cpu@1 missing clock-frequency property
    [    0.002174] /cpus/cpu@2 missing clock-frequency property
    [    0.002220] /cpus/cpu@3 missing clock-frequency property
    [    0.002236] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
    [    0.002277] Setting up static identity map for 0x800082c0 - 0x800083cc
    [    0.069590] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
    [    0.089630] CPU2: thread -1, cpu 2, socket 0, mpidr 80000002
    [    0.109688] CPU3: thread -1, cpu 3, socket 0, mpidr 80000003
    [    0.109835] Brought up 4 CPUs
    [    0.109866] SMP: Total of 4 processors activated (819.20 BogoMIPS).
    [    0.109877] CPU: All CPU(s) started in SVC mode.
    [    0.110599] devtmpfs: initialized
    [    0.128362] VFP support v0.3: implementor 41 architecture 4 part 30 variant f rev 0
    [    0.129013] clocksource jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
    [    0.130270] pinctrl core: initialized pinctrl subsystem
    [    0.131736] NET: Registered protocol family 16
    [    0.133291] DMA: preallocated 256 KiB pool for atomic coherent allocations
    [    0.151828] irq: no irq domain found for /soc/keystone_irq@26202a0 !
    [    0.152308] irq: no irq domain found for /soc/keystone_irq@26202a0 !
    [    0.152780] irq: no irq domain found for /soc/keystone_irq@26202a0 !
    [    0.153253] irq: no irq domain found for /soc/keystone_irq@26202a0 !
    [    0.153726] irq: no irq domain found for /soc/keystone_irq@26202a0 !
    [    0.154209] irq: no irq domain found for /soc/keystone_irq@26202a0 !
    [    0.154689] irq: no irq domain found for /soc/keystone_irq@26202a0 !
    [    0.155158] irq: no irq domain found for /soc/keystone_irq@26202a0 !
    [    0.168425] hw-breakpoint: found 5 (+1 reserved) breakpoint and 4 watchpoint registers.
    [    0.168440] hw-breakpoint: maximum watchpoint size is 8 bytes.
    [    0.189159] vgaarb: loaded
    [    0.189769] SCSI subsystem initialized
    [    0.190726] usbcore: registered new interface driver usbfs
    [    0.190910] usbcore: registered new interface driver hub
    [    0.191126] usbcore: registered new device driver usb
    [    0.193780] pps_core: LinuxPPS API ver. 1 registered
    [    0.193794] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
    [    0.193923] PTP clock support registered
    [    0.196211] Switched to clocksource arch_sys_counter
    [    0.219575] NET: Registered protocol family 2
    [    0.220610] TCP established hash table entries: 4096 (order: 2, 16384 bytes)
    [    0.220675] TCP bind hash table entries: 4096 (order: 3, 32768 bytes)
    [    0.220787] TCP: Hash tables configured (established 4096 bind 4096)
    [    0.220850] UDP hash table entries: 256 (order: 1, 8192 bytes)
    [    0.220883] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
    [    0.221128] NET: Registered protocol family 1
    [    0.221609] RPC: Registered named UNIX socket transport module.
    [    0.221624] RPC: Registered udp transport module.
    [    0.221636] RPC: Registered tcp transport module.
    [    0.221647] RPC: Registered tcp NFSv4.1 backchannel transport module.
    [    0.222234] CPU PMU: Failed to parse /pmu/interrupt-affinity[0]
    [    0.222289] hw perfevents: enabled with armv7_cortex_a15 PMU driver, 7 counters available
    [    0.223861] platform alarmtimer: set dma_pfn_offset00780000
    [    0.224804] futex hash table entries: 16 (order: -2, 1024 bytes)
    [    0.239077] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
    [    0.239268] ntfs: driver 2.1.32 [Flags: R/O].
    [    0.239705] jffs2: version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
    [    0.242411] NET: Registered protocol family 38
    [    0.242741] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
    [    0.242765] io scheduler noop registered
    [    0.242784] io scheduler deadline registered
    [    0.242838] io scheduler cfq registered (default)
    [    0.243084] keystone_irq soc:keystone_irq@26202a0: irqchip registered, nr_irqs 28
    [    0.246754] keystone-pcie 21801000.pcie: No pcie serdes driver found: -38
    [    0.246795] keystone-pcie: probe of 21801000.pcie failed with error -38
    [    0.247207] keystone-navigator-qmss soc:qmss@2a40000: qmgr start queue 0, number of queues 8192
    [    0.247452] keystone-navigator-qmss soc:qmss@2a40000: added qmgr start queue 0, num of queues 8192, reg_peek e0980000, reg_status e09a2000, reg_config e09a4000, reg_region e09a6000, reg_push e09c0000, reg_pop e0a00000
    [    0.247478] keystone-navigator-qmss soc:qmss@2a40000: qmgr start queue 8192, number of queues 8192
    [    0.247698] keystone-navigator-qmss soc:qmss@2a40000: added qmgr start queue 8192, num of queues 8192, reg_peek e0a40000, reg_status e09a8400, reg_config e09aa000, reg_region e09ac000, reg_push e0a80000, reg_pop e0ac0000
    [    0.247951] keystone-navigator-qmss soc:qmss@2a40000: failed to get firmware for pdsp
    [    0.249415] keystone-navigator-qmss soc:qmss@2a40000: pdsp id 0 not started for range acc-low-0
    [    0.250613] keystone-navigator-dma soc:knav_dmas@0: DMA dma_gbe registered 41 logical channels, flows 32, tx chans: 9, rx chans: 24
    [    0.250834] keystone-navigator-dma soc:knav_dmas@0: DMA dma_xgbe registered 48 logical channels, flows 32, tx chans: 16, rx chans: 16
    [    0.360917] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
    [    0.361080] platform serial8250: set dma_pfn_offset00780000
    [    0.363864] console [ttyS0] disabled
    [    0.363956] 2530c00.serial: ttyS0 at MMIO 0x2530c00 (irq = 26, base_baud = 6400000) is a 16550A
    [    1.231591] console [ttyS0] enabled
    [    1.236446] 2531000.serial: ttyS1 at MMIO 0x2531000 (irq = 27, base_baud = 6400000) is a 16550A
    [    1.259483] brd: module loaded
    [    1.270558] loop: module loaded
    [    1.274029] at24 0-0050: 131072 byte 24c1024 EEPROM, writable, 1 bytes/write
    [    1.283787] spi_davinci 21000400.spi: Controller at 0xe0a3c400
    [    1.290404] spi_davinci 21000600.spi: Controller at 0xe0a3e600
    [    1.297013] spi_davinci 21000800.spi: Controller at 0xe0a7c800
    [    1.346263] davinci_mdio 2090300.mdio: davinci mdio revision 1.5
    [    1.352322] libphy: 2090300.mdio: probed
    [    1.359123] netcp-1.0 2620110.netcp: module(netcp-xgbe) not used for device
    [    1.366357] 2620738.usb_phy supply vcc not found, using dummy regulator
    [    1.774908] platform xhci-hcd.0.auto: set dma_pfn_offset00780000
    [    1.782223] xhci-hcd xhci-hcd.0.auto: xHCI Host Controller
    [    1.788078] xhci-hcd xhci-hcd.0.auto: new USB bus registered, assigned bus number 1
    [    1.796393] xhci-hcd xhci-hcd.0.auto: hcc params 0x0298f06d hci version 0x100 quirks 0x00210010
    [    1.805186] xhci-hcd xhci-hcd.0.auto: irq 34, io mem 0x02690000
    [    1.811462] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
    [    1.818336] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
    [    1.825620] usb usb1: Product: xHCI Host Controller
    [    1.830560] usb usb1: Manufacturer: Linux 4.1.13-g8dc6617 xhci-hcd
    [    1.836811] usb usb1: SerialNumber: xhci-hcd.0.auto
    [    1.842601] hub 1-0:1.0: USB hub found
    [    1.846463] hub 1-0:1.0: 1 port detected
    [    1.850984] xhci-hcd xhci-hcd.0.auto: xHCI Host Controller
    [    1.856865] xhci-hcd xhci-hcd.0.auto: new USB bus registered, assigned bus number 2
    [    1.864711] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
    [    1.873123] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003
    [    1.879996] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
    [    1.887301] usb usb2: Product: xHCI Host Controller
    [    1.892220] usb usb2: Manufacturer: Linux 4.1.13-g8dc6617 xhci-hcd
    [    1.898472] usb usb2: SerialNumber: xhci-hcd.0.auto
    [    1.904241] hub 2-0:1.0: USB hub found
    [    1.908097] hub 2-0:1.0: 1 port detected
    [    1.912852] usbcore: registered new interface driver usb-storage
    [    1.919435] mousedev: PS/2 mouse device common for all mice
    [    1.925484] i2c /dev entries driver
    [    1.930720] davinci-wdt 22f0080.wdt: heartbeat 60 sec
    [    1.963579] keystone-crypto 20c0000.crypto: crypto accelerator enabled
    [    1.970711] usbcore: registered new interface driver usbhid
    [    1.976356] usbhid: USB HID core driver
    [    1.982403] nand: device found, Manufacturer ID: 0x2c, Chip ID: 0xac
    [    1.988837] nand: Micron MT29F4G08ABBDAHC
    [    1.992884] nand: 512 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
    [    2.000965] Bad block table found at page 262080, version 0x01
    [    2.007804] Bad block table found at page 262016, version 0x01
    [    2.014178] 3 ofpart partitions found on MTD device 30000000.nand
    [    2.020348] Creating 3 MTD partitions on "30000000.nand":
    [    2.025794] 0x000000000000-0x000000100000 : "u-boot"
    [    2.032231] 0x000000100000-0x000000180000 : "params"
    [    2.038622] 0x000000180000-0x000020000000 : "ubifs"
    [    2.045727] davinci_nand 30000000.nand: controller rev. 2.5
    [    2.051906] platform oprofile-perf.0: set dma_pfn_offset00780000
    [    2.058342] oprofile: using timer interrupt.
    [    2.062918] ipip: IPv4 over IPv4 tunneling driver
    [    2.068522] gre: GRE over IPv4 demultiplexor driver
    [    2.073570] Initializing XFRM netlink socket
    [    2.077941] NET: Registered protocol family 17
    [    2.082459] NET: Registered protocol family 15
    [    2.087091] Registering SWP/SWPB emulation handler
    [    2.106757] pps pps0: new PPS source ptp0
    [    2.110841] cpts rftclk: sel(1),rate(204800000 HZ),mult(2560),shift(10),div(1)
    [    2.136424] IP-Config: Complete:
    [    2.139689]      device=eth0, hwaddr=d4:f5:13:aa:d8:ba, ipaddr=172.17.1.57, mask=255.255.0.0, gw=172.17.0.3
    [    2.149537]      host=172.17.1.57, domain=, nis-domain=(none)
    [    2.155331]      bootserver=172.17.1.146, rootserver=172.17.1.146, rootpath=
    [    2.261608] VFS: Mounted root (nfs filesystem) on device 0:14.
    [    2.268212] devtmpfs: mounted
    [    2.271467] Freeing unused kernel memory: 248K (c069c000 - c06da000)
    
    INIT: version 2.88 booting
    
    [    3.253997] NET: Registered protocol family 41
    Starting random number generator daemonread error
    
    
    
    read error
    
    
    
    .
    
    [    3.591770] random: dd urandom read with 106 bits of entropy available
    Sat Jan  9 00:41:53 UTC 2016
    
    
    INIT: Entering runlevel: 5
    
    
    [    4.561308] random: nonblocking pool is initialized
    NOT configuring network interfaces: / is an NFS mount
    
    udhcpc (v1.23.1) started
    
    Sending discover...
    
    Sending select for 172.17.2.177...
    
    Lease of 172.17.2.177 obtained, lease time 7200
    
    /etc/udhcpc.d/50default: Adding DNS 172.17.0.1
    
    /etc/udhcpc.d/50default: Adding DNS 8.8.8.8
    
    /etc/udhcpc.d/50default: Adding DNS 8.8.4.4
    
    /etc/udhcpc.d/50default: Adding DNS 202.251.202.3
    
    Starting system message bus: dbus.
    
    Starting Dropbear SSH server: [    6.954200] NET: Registered protocol family 10
    dropbear.
    
    Starting mpmsrv daemon.
    
    platform: k2hk-evm
    
    Starting softhsm daemon.
    
    Starting telnet daemon.
    
    Starting tiipclad daemon
    
    GateMP is not supported for this device
    
    
    
    Opened log file: lad.txt
    
    
    
    numProcessors = 9 id = 0 baseId = 0
    
    
    
    Spawned daemon: /usr/bin/lad_tci6638
    
    
    
    .
    
    Starting rpcbind daemon...done.
    
    starting statd: done
    
    Starting thttpd.
    
    Starting Lighttpd Web Server: lighttpd.
    
    2016-01-09 00:41:56: (/home/gtbldadm/processor-sdk-linux-fido-build/build-CORTEX_1/arago-tmp-external-linaro-toolchain/work/cortexa15hf-vfp-neon-linux-gnueabi/lighttpd/1.4.35-r1/lighttpd-1.4.35/src/log.c.164) server started 
    
    * starting FTP Server: vsftpd... done.
    
     
    
     
    
    #>>>>> LCD 12
    
    IP Address:
    
    172.17.1.57
    
    Enabling thermal zones...
    
    /etc/rc5.d/S98thermal-zone-init: line 7: /sys/class/thermal/thermal_zone*/mode: No such file or directory
    
    ***************************************************************
    
    ***************************************************************
    
    NOTICE: This file system contains the following GPLv3 packages:
    
    autoconf
    
    binutils-dev
    
    binutils
    
    bison-dev
    
    bison
    
    cpp-symlinks
    
    cpp
    
    g++-symlinks
    
    g++
    
    gawk-dev
    
    gawk
    
    gcc-symlinks
    
    gcc
    
    gdb
    
    gdbc6x
    
    gdbserver
    
    libgmp10
    
    libmpc3
    
    libmpfr4
    
    make
    
    
    
    If you do not wish to distribute GPLv3 components please remove
    
    the above packages prior to distribution.  This can be done using
    
    the opkg remove command.  i.e.:
    
        opkg remove <package>
    
    Where <package> is the name printed in the list above
    
    
    
    NOTE: If the package is a dependency of another package you
    
          will be notified of the dependent packages.  You should
    
          use the --force-removal-of-dependent-packages option to
    
          also remove the dependent packages as well
    
    ***************************************************************
    
    ***************************************************************
    
    
    
     _____                    _____           _         _   
    
    |  _  |___ ___ ___ ___   |  _  |___ ___  |_|___ ___| |_ 
    
    |     |  _| .'| . | . |  |   __|  _| . | | | -_|  _|  _|
    
    |__|__|_| |__,|_  |___|  |__|  |_| |___|_| |___|___|_|  
    
                  |___|                    |___|            
    
    
    
    Arago Project http://arago-project.org k2hk-evm /dev/ttyS0
    
    
    
    Arago 2015.12 k2hk-evm /dev/ttyS0
    
    
    
    k2hk-evm login: root
    root@k2hk-evm:~# cd /
    root@k2hk-evm:/# ls
    bin                                   mpmsrv_keystone2_example_orginal.out
    boot                                  opt
    dev                                   proc
    etc                                   run
    home                                  sbin
    include                               srv
    ipaddr                                sys
    lib                                   tmp
    media                                 usr
    mnt                                   var
    mpmsrv_keystone2_example.out          www
    root@k2hk-evm:/# 
    root@k2hk-evm:/# 
    root@k2hk-evm:/# 
    root@k2hk-evm:/# lsmod
    Module                  Size  Used by
    ipv6                  339102  30 
    rpmsg_proto             6722  0 
    virtio_rpmsg_bus       11706  1 rpmsg_proto
    remoteproc             25173  1 rpmsg_proto
    virtio                  6432  2 remoteproc,virtio_rpmsg_bus
    virtio_ring            12981  2 remoteproc,virtio_rpmsg_bus
    root@k2hk-evm:/#  
    root@k2hk-evm:/# 
    root@k2hk-evm:/# 
    root@k2hk-evm:/# 
    root@k2hk-evm:/# 
    root@k2hk-evm:/#             
    root@k2hk-evm:/# 
    root@k2hk-evm:/# insmod lib/modules/k 4.1.13-g8dc6617/kernel/drivers/uio/uio
    uio.ko              uio_dmem_genirq.ko  uio_pdrv_genirq.ko
    root@k2hk-evm:/# insmod lib/modules/4.1.13-g8dc6617/kernel/drivers/uio/uio.ko 
    root@k2hk-evm:/# insmod lib/modules/4.1.13-g8dc6617/kernel/drivers/remoteproc/keystone_remoteproc.ko 
    [   72.997787] keystone-rproc 10800000.dsp0: assigned reserved memory node dsp_common_cma_pool
    [   73.006343]  remoteproc0: 10800000.dsp0 is available
    [   73.011351]  remoteproc0: Note: remoteproc is still under development and considered experimental.
    [   73.020661]  remoteproc0: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
    [   73.035804] keystone-rproc 11800000.dsp1: assigned reserved memory node dsp_common_cma_pool
    [   73.044342]  remoteproc1: 11800000.dsp1 is available
    [   73.049777]  remoteproc1: Note: remoteproc is still under development and considered experimental.
    [   73.059136]  remoteproc1: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
    [   73.073219] keystone-rproc 12800000.dsp2: assigned reserved memory node dsp_common_cma_pool
    [   73.082045]  remoteproc2: 12800000.dsp2 is available
    [   73.087363]  remoteproc2: Note: remoteproc is still under development and considered experimental.
    [   73.096444]  remoteproc2: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
    [   73.110238] keystone-rproc 13800000.dsp3: assigned reserved memory node dsp_common_cma_pool
    [   73.118778]  remoteproc3: 13800000.dsp3 is available
    [   73.123786]  remoteproc3: Note: remoteproc is still under development and considered experimental.
    [   73.132848]  remoteproc3: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
    [   73.146692] keystone-rproc 14800000.dsp4: assigned reserved memory node dsp_common_cma_pool
    [   73.155190]  remoteproc4: 14800000.dsp4 is available
    [   73.160230]  remoteproc4: Note: remoteproc is still under development and considered experimental.
    [   73.169291]  remoteproc4: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
    [   73.183034] keystone-rproc 15800000.dsp5: assigned reserved memory`
    
    U-Boot 2015.07-00080-g68db1b0-dirty (May 12 2016 - 11:50:43 +0530)
    
    CPU: 66AK2Hx SR2.0
    I2C:   ready
    DRAM:  DDR3 PLL (PLL2) Setup ... 
    DDR3 PLL Setup complete, DDR3A clock now running at 400MHz.
    DDR3A initialization complete 
    1 GiB
    NAND:  512 MiB
    Net:   K2HK_EMAC
    Hit any key to stop autoboot:  1  0 
    U-Boot# 
    U-Boot# 

    Complete bootlog attached.

  • Have you disabled the udev ?
    Can you please enable the udev and see?
    Udev will load and run the kernel module driver automatically.
  • I tried that too,

    I booted udev manually and looks unstable when it is trying to load  keystone-dsp-mem

    Why it is not able to load this driver ,any dependency is missing ???

    root@k2hk-evm:/# lsmod
    Module                  Size  Used by
    ipv6                  339102  30
    rpmsg_proto             6722  0
    virtio_rpmsg_bus       11706  1 rpmsg_proto
    remoteproc             25173  1 rpmsg_proto
    virtio                  6432  2 remoteproc,virtio_rpmsg_bus
    virtio_ring            12981  2 remoteproc,virtio_rpmsg_bus
    root@k2hk-evm:/#
    root@k2hk-evm:/#
    root@k2hk-evm:/#
    root@k2hk-evm:/# /etc/init.d/udev
    -sh: /etc/init.d/udev: No such file or directory
    root@k2hk-evm:/# /etc/init.d/udev status
    -sh: /etc/init.d/udev: No such file or directory
    root@k2hk-evm:/# /etc/init.d/udevbk status
    udevd is stopped
    root@k2hk-evm:/# /etc/init.d/udevbk start
    Starting udev
    udev: Not using udev cache because of changes detected in the following files:
    udev:     /proc/version /proc/cmdline /proc/devices
    udev:     lib/udev/rules.d/* etc/udev/rules.d/*
    udev: The udev cache will be regenerated. To identify the detected changes,
    udev: compare the cached sysconf at   /etc/udev/cache.data
    udev: against the current sysconf at  /dev/shm/udev.cache
    [   57.794117] udevd[1599]: starting version 182
    [   58.202028] keystone-rproc 10800000.dsp0: assigned reserved memory node dsp_common_cma_pool
    [   58.388874]  remoteproc0: 10800000.dsp0 is available
    [   58.457962]  remoteproc0: Note: remoteproc is still under development and considered experimental.
    [   58.643566]  remoteproc0: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
    [   58.908905] keystone-dsp-mem c000000.dspmem: registered misc device dspmem
    [   59.039762] keystone-rproc 11800000.dsp1: assigned reserved memory node dsp_common_cma_pool
    [   59.222117]  remoteproc1: 11800000.dsp1 is available
    [   59.351837]  remoteproc1: Note: remoteproc is still under development and considered experimental.
    [   59.564428]  remoteproc1: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
    [   59.744806] keystone-rproc 12800000.dsp2: assigned reserved memory node dsp_common_cma_pool
    [   60.070266]  remoteproc2: 12800000.dsp2 is available
    [   60.248070]  remoteproc2: Note: remoteproc is still under development and considered experimental.
    [   60.585107]  remoteproc2: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
    [   60.982186] keystone-rproc 13800000.dsp3: assigned reserved memory node dsp_common_cma_pool
    [   61.292079]  remoteproc3: 13800000.dsp3 is available
    [   61.317843]  remoteproc3: Note: remoteproc is still under development and considered experimental.
    [   61.370586]  remoteproc3: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
    [   61.434985] keystone-rproc 14800000.dsp4: assigned reserved memory node dsp_common_cma_pool
    [   61.502998]  remoteproc4: 14800000.dsp4 is available

  • Titus,


    When i load keystone_remoteproc.ko module, board get restarted.

    Is this sdk working at your end.

  • I have analyzed some scenarios, hope this will help you to give us some inputs,

    Instead of mounting udev, i tried to mount the modules manually like below,

    insmod lib/modules/4.1.13-g8dc6617/kernel/drivers/soc/ti/keystonne_dsp_mem.ko
    
    insmod lib/modules/4.1.13-g8dc6617/kernel/drivers/uio/uio.ko
    
    insmod lib/modules/4.1.13-g8dc6617/kernel/drivers/remoteproc/keystone_remoteproc.ko

    and listed lsmod command to see the modules,

    root@k2hk-evm:/# lsmod
    Module                  Size  Used by
    keystone_remoteproc     9066  0
    uio                     8891  1 keystone_remoteproc
    keystone_dsp_mem        2907  0
    rpmsg_proto             6823  0
    virtio_rpmsg_bus       11766  1 rpmsg_proto
    remoteproc             25480  2 rpmsg_proto,keystone_remoteproc
    virtio                  6460  2 remoteproc,virtio_rpmsg_bus
    virtio_ring            12987  2 remoteproc,virtio_rpmsg_bus
    ipv6                  338366  30

    Once module is loaded,i tried to do mpmcl to load binaries to slave dsp's

    check the status of dsp0,

    root@k2hk-evm:/# mpmcl status dsp0
    dsp0 is in idle state

    check the ping status

    root@k2hk-evm:/# mpmcl ping dsp0
    ping succeeded


    then load mpmsrv_keystone2_example.out, first i get error msg and second time if i load , it is loading successfully,

    root@k2hk-evm:/# mpmcl load dsp0 mpmsrv_keystone2_example.out
    load failed (error: -104)
    root@k2hk-evm:/# mpmcl load dsp0 mpmsrv_keystone2_example.out
    load succeeded

    root@k2hk-evm:/# mpmcl run dsp0
    run succeeded

    Because of first time error, dsp0 goes to error state, even if i give reset command its still in error state and reset command is also not successful,

    root@k2hk-evm:/# mpmcl reset dsp0
    reset error

    I guess because of this error state i am not able to see any trace logs in remoteproc directory,

    Hope these inputs will help,

    Please suggest,