This thread has been locked.
If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.
We're using DM6467T DVSDK 3.10 GA. It has served us very well.
We have an audio driver that uses DMA from a McASP to read samples from an AIC32. This has appeared to be rock solid for a long time. Now that we have hundreds of units in the field, though, we're getting an occasional ARM hang (as in nothing else happens on any interfaces - UART, PCI network driver, etc.). By digging into the ARM's DDR2 over PCI I can see the final dmesg output by following the crumbs from log_buf in System.map. The hang happens when the audio driver starts the McASP. Meanwhile a user thread is creating the code and that thread dies when the Codec Server reaches out to the DSP (see http://e2e.ti.com/support/embedded/f/354/t/140023.aspx "ARM hangs after Comm_create during instantiation of DSP codec"). As indicated in that posting, this problem is usually is repeatable once it gets in that state. It goes away after a power cycle.
When I saw http://e2e.ti.com/support/embedded/f/354/p/104964/504621.aspx ("DM6467T Linux driver - how to use EDMA") I realized that I may need to reserve more DMA channels for the DSP. It seems like a likely candidate to solve my problem. But not all of the questions were answered there. I believe this happened because it was a "Verified Answer" before all the original poster's questions were answered (and I added a few of my own!).
So I'd just like to know how to determine which DMA channels are in use by the DSP, and how to avoid them (presumably via entries in dm646x_dma_rsv_chans in the ARM Linux kernel source, arch/arm/mach-davinci/dm646x.c).
I've looked at a number of places to try to determine which DMA channels I should reserve in the ARM kernel to make sure that my own driver's use of EDMA does not interfere with the ones used by the Codec Server running in the DSP. I could simply reserve the first 49 and hope that my problem goes away. But I would like the confidence to tell management that the problem was solved; it may appear two weeks later in a customer's unit. Not good.
A quick summary of the places I've looked:
So my questions are:
Thanks!
Doug
After re-reading the response in "DM6467T Linux driver - how to use EDMA", I dug more into the EDMA3 LLD library. In dvsdk_3_10_00_19/edma3_lld_01_11_00_03/packages/ti/sdo/edma3/rm/docs/EDMA3_RM_Datasheet.pdf, there's the following paragraph:
For easy and quick reconfiguration of driver for different SoCs, all its global configurable parameters can be passed at run time to the API EDMA3_RM_create (), to create the SoC specific EDMA3 Resource Manager Object. In case this configuration is not passed at run time, it can be taken from the EDMA3 configuration file edma3_<PLATFORM_NAME>_cfg.c, for the specific SoC, if it has been provided there. The configuration files can be found in “edma3_lld_<VERSION_NUMBER>\packages\ti\sdo\edma3\rm\src\configs\” folder.
Sure enough, rm/src/configs/edma3_dm646x_cfg.c contains a bit-mask of DMA channels, TCCs, and parameter RAM slots. Here's the important bit of code in defInstInitConfig:
{
/* Resources owned by Region 1 */
/* ownPaRAMSets */
{0x0u, 0x0u, 0x0u, 0x0u,
0xFFFFFFFFu, 0xFFFFFFFFu, 0xFFFFFFFFu, 0xFFFFFFFFu,
0xFFFFFFFFu, 0xFFFFFFFFu, 0xFFFFFFFFu, 0xFFFFFFFFu,
0xFFFFFFFFu, 0xFFFFFFFFu, 0xFFFFFFFFu, 0xFFFFFFFFu},
/* ownDmaChannels */
{0xFFF0E00Fu, 0xFF3FFFFFu},
/* ownQdmaChannels */
{0x0000007Fu},
/* ownTccs */
{0xFFF0E00Fu, 0xFF3FFFFFu},
Note how this is not a complete match for the code in dm646x.c in the ARM kernel. Here are the reservations in dm646x.c:
static const s8 dm646x_dma_rsv_chans[][2] = {
/* (offset, length) */
{ 0, 4},
{13, 3},
{24, 4},
{30, 2},
{54, 3},
{-1, -1}
};
static const s16 dm646x_dma_rsv_slots[][2] = {
/* (offset, length) */
{ 0, 4},
{13, 3},
{24, 4},
{30, 2},
{54, 3},
{128, 384},
{-1, -1}
};
I wish the writer of dm646x.c had included comments about why these particular reservations are made. rsv_chans starts off with the proper values { 0, 4 } and { 13, 3}, but shouldn't the next one be { 24, 12 }? The last rsv_slots { 128, 384 } clearly matches ownPaRAMSets, and I assume the others are meant to hold the one-to-one mapping from channel to initial slot in rsv_chans.
Maybe this was corrected in updates to the software which I did not track. Or maybe I'm off base. Either way, I'd love to hear from a TI rep about the subject.
Hello TI experts,
I see that you have moved this post back and forth between hardware and software. If it is in the wrong forum that is fine but it has also been open without a response for over a week. These details are important to developing a robust system that will not crash in the field. Please add your input as soon as possible.
Thanks,
Brandy
I'd like to know about EDMA allocation betweem ARM and DSP as well. My processor is the OMAP-L137. In my case, some of the mystery DMA numbers in
linux-03.21.00.04/arch/arm/mach-davinci/board-da830.c
edma3_lld_01_11_01_04\packages\ti\sdo\edma3\drv\sample\src\configs\bios_edma3_drv_sample_c6747_cfg.c
seem to match with channel event numbers in
pspdrivers_01_30_01\packages\ti\pspiom\cslr\soc_C6747.h
which are also in the processor datasheet in a table called "EDMA Events". Still a lot of other numbers to be deciphered.
If I am interpreting the numbers correctly, the DSP reserves a lot EDMA channels that it might not use, eg UARTs SPIs, etc. The question is whether EDMA channels can be released from the DSP side to allow the ARM to use them? Does DSPLink use any EDMA channels? In my case, comments in cfg.c imply that 22 and 23 are used for memory to memory copy.
I am wondering if this might be related to spurious EDMA interrupts that I get on the ARM side when the DSP is executing. Some EDMA channel that needs to be reserved for the DSP?
I modified the DMA resource reservations in dm646x.c to include the EDMA resources I found in edma3_dm646x_cfg.c under edma3_lld. My problem still occurred, so there's something else going on. It could still be a hardware issue.
But I still would like an answer to my original three questions:
FYI, here is the mod I made to dm646x.c:
/*----------------------------------------------------------------------*/
/*
* From defInstInitConfig in edma3_lld_01_11_00_03/packages/
* ti/sdo/edma3/rm/src/configs/edma3_dm646x_cfg.c:
*
* /\* Resources owned by Region 1 *\/
* /\* ownPaRAMSets *\/
* {0x0u, 0x0u, 0x0u, 0x0u,
* 0xFFFFFFFFu, 0xFFFFFFFFu, 0xFFFFFFFFu, 0xFFFFFFFFu,
* 0xFFFFFFFFu, 0xFFFFFFFFu, 0xFFFFFFFFu, 0xFFFFFFFFu,
* 0xFFFFFFFFu, 0xFFFFFFFFu, 0xFFFFFFFFu, 0xFFFFFFFFu},
*
* /\* ownDmaChannels *\/
* {0xFFF0E00Fu, 0xFF3FFFFFu},
*
* /\* ownQdmaChannels *\/
* {0x0000007Fu},
*
* /\* ownTccs *\/
* {0xFFF0E00Fu, 0xFF3FFFFFu},
*
*/
static const s8 dm646x_dma_rsv_chans[][2] = {
/* (offset, length) */
{ 0, 4},
{13, 3}, /* Available before: 5-12 */
{24, 8}, /* Available before: 16-23 */
{32, 22},
{56, 8}, /* Available before: 54-55 */
{-1, -1}
};
/*
* See the note above.
*/
static const s16 dm646x_dma_rsv_slots[][2] = {
/* (offset, length) */
{ 0, 4},
{13, 3}, /* Available before: 5-12 */
{24, 8}, /* Available before: 16-23 */
{32, 22},
{56, 8}, /* Available before: 54-55 */
{128, 384}, /* Available before: 65-127 */
{-1, -1}
};
Doug,
Since EDMA3 is a shared resource, the DMA/QDMA channels and param sets are allocated to both ARM & DSP (in a multi-core environment) as per the software requirements and some additional buffers. Software modules like device drivers, codec servers etc know their static and run time requirements and accordingly, EDMA3 resources are distributed. Resources can be overlapping or mutually exclusive, as suggested by EDMA3 users, and may not be bit-by-bit same in the EDMA3 configuration files which you are referring to.
Regarding the channels used by codec server and others, you can search for the APIs which request dma channels like EDMA3_DRV_requestChannel() in LLD and edma_alloc_channel() in dma.c and try to find out the channels requested by them.
Anuj
Anuj,
Thank you for you response. I had already tracked down the edma_alloc_channel() calls on the ARM side. I'm not too worried about that because I've been working in that area quite a bit and all the code makes sense and uses the resource reservation system that's already in place. What I'm worried about is the DSP side.
The DSP code has always been somewhat of a black box, handled by the Codec Engine. I've done an exhaustive search of the C code in the entire DVSDK and I have found no calls to EDMA3_DRV_requestChannel() outside the EDMA3_LLD library examples. I used:
find "/path/to/dvsdk_3_10_00_19/" \( \( -name .svn -prune \) -o \( -name "*.c" -o -name "*.c++" -o -name "*.cpp" -o -name "*.cc" \) \) -exec grep -i -n "edma3_drv_request" /dev/null {} \;
So no hits there.
We do not get the source code to the codecs themselves. There's a lot of wrapper code for them that uses DMA channels. In particular, h264fhdvenc (the codec we're using) has RMAN configuration structures in h264fhdvenc/app/Client/Test/Src/rman_config.c. Some are #if 0'd out, others are in use. In particular, see EDMA3_EdmaChanScratchSizes and EDMA3_QdmaChanScratchSizes. Those indicate 43 EDMA and 7 QDMA channels. I think. Some of the values in that file are well-commented. Others not so much.
I was surprised to see that there was no mention of the number of DMA channels being used by the codec in its User's Manual. There is mention of DMA (if it's used, then DMAN3_init() must be called first), but nothing about how many channels are needed.
I did one other thing:
cd dvsdk_3_10_00_19/cs2dm6467_1_00_00_10/packages/ti/sdo/codecs/h264fhdvenc/lib
strings h264fhdvenc_tii.l64P | grep -i dma
There's a lot of code in the codec to perform DMA.
I'm disappointed that it takes so much effort on my part to track down these numbers. It seems like I'm being told by TI that it's my responsibility to make sure that the shared resources are not in conflict. I get that, and I understand that it will be application-specific. I will shoulder that burden, to an extent. But TI has to do its part.
There's SO much code that it is unreasonable to expect your customers to become intimately familiar with each piece. I think a reasonable compromise would have been to have TI list the numbers for each of their software components, like they often do with memory consumption. At a minimum I would have expected them to be in the User's Guide for the codecs. Maybe in future releases this can be done?
I've Googled and searched the DVSDK. But maybe I'm just missing a document somewhere. Is there a document that lists where to look for these DMA resources?
If you have any other ideas, please post them here. If you would like me to search for a particular thing, please check once yourself to make sure they really exist.
My questions remain (and I've added two):
Anuj, I'm happy you replied. I was starting to feel unloved. I've been generally happy with TI's responsiveness and with their code quality. Thank you!
Dough,
The Codecs do not ask for specific channels (eg CH0 or CH1 or CH2). The codecs (via Codec Server) ask for the number of DMA channels and Paramsets. In general the DM6467 codecs do not use QDMA channels, only EDMA channesl are used. So for example the Codec might ask 24 EDMA channesl and 134 Paramsets. It will not ask that only CH0 or CH1 be allocated to it, or Paramset 54 be allocated. What the codec is looking for is the correct type of DMA channel and the correct number of channels to be allocated to it.
Regards,
Kapil
Kapil,
Who gives the codec permission to use whatever channels? If it just needs an amount of channels, who controls that it is not using the same channels as the ARM? Is there a shared resource manager that I am not finding?
In other words, is the ARM delegating out all channels to both the DSP and linux drivers or does the DSP check the register settings before it allocates itself a channel to make sure the ARM has not already taken that channel?
Our point is, the documentation states that the developer is responsible for making sure that the DSP and ARM DMA channels are being shared nicely. Unfortunately we did not write the codec and we can't find any documenation discussing how the codec is alloted channels. Without this information we can not take responsibility to make sure the ARM is not using the same channels as the DSP or that the DSP is not taking channels that the ARM is already using.
How are the resources shared nicely in your applications? How do you know that the ARM and DSP are not using the same DMA channels? How can I check my application to make sure too?
Thanks for the help,
Brandy
Thanks for the information, Kapil. That is exactly how I thought it worked - though I'm happy to see that QDMA channels are not used. That means I don't need to worry about conflicts in QDMA allocation.
But, as Brandy pointed out, the main question still remains. How do the ARM and DSP shared this set of resources?
I would appreciate it immensely if you (or another rep) could reply to the numbered questions I've asked. It may be that the same answer applies to multiple questions, but my whole reason for posting them in a numbered fashion was to encourage answers to those specific questions.
Thanks!
Hi,
The EDMA3 LLD acts as a resource manager (RM) on the DSP for all EDMA resources. The config structure you have mentioned in earlier posts show the PARAMS, TCCs, EDMA and QDMA channels that are available for allocation on the DSP side.
At runtime "drivers" acquire resources directly from the LLD RM via EDMA3_DRV_requestChannel calls, which assigns from the available pool.
Codecs however use a special resource management interface (IRES_EDMA3CHAN) to publish their EDMA requirements. During codec instance creation Codec Engine (CE) uses RMAN (a resource manager handling IRES type resource requests) to inquire/allocate and grant the requested resources to each codec.
RMAN uses the LLD RM also to actually allocate and free the EDMA resources, so the configuration of the EDMA3 LLD still stands and gets honored on the DSP.
While drivers may need specific TCCs and EDMA channels, codecs typically require "ANY" TCC, EDMA or PARAM. By obtaining a CE and FC trace on the DSP you can see the specific tcc, param and edma channel that actually got assigned to each algorithm instance (even though the trace may be a bit verbose.) See these links to enable debug/trace:
http://processors.wiki.ti.com/index.php/Trace_in_Framework_Components
http://processors.wiki.ti.com/index.php/CE_DEBUG
But first check for the specific EDMA resources that your ARM McASP audio driver is using and make sure that is excluded from the DSP side LLD configuration (by '0'ing the corresponding bits in the bitmasks.)
If you don't see any resource conflicts but still see hang occuring then you may need to dig into the hardware errata you mentioned as well. Specifically, the one:
Advisory 3.0.3 —DSP SDMA/IDMA: Unexpected Stalling and Potential Deadlock Condition When DSP L2 Memory Ports Used as RAM When L2 Memory Configured as Non-cache (RAM)
has bitten a few. This may mean checking with your codec vendor whether they have taken care of this issue.
Best regards,
Murat
Thank you, Murat. That's an excellent reply and I appreciate it. Especially important is the knowledge that a few other developers have stumbled into Advisory 3.0.3. Now I'll take the time to understand its intricacies.
Yes, I have modified my ARM kernel initialization structures and I believe that I have protected myself from DMA resource conflicts. I was hoping for confirmation from TI that the steps I have taken, all mentioned in this thread, are sufficient to prevent such a resource conflict.
The codec I'm using is h264fhdvenc 01.10.02.03, the version that shipped with DM6467T DVSDK 3.10 GA. This is provided by TI (though I have heard that TI did not write it). So you are my codec vendor! The User's Guide (SPRUGN8D) says nothing about the erratum and nothing relevant about L2 cache, internal memory or RAM.
So two new questions arise from your post:
(6) Is h264fhdvenc 01.10.02.03 (and each of the other codecs in DVSDK 3_10_00_19 GA) susceptible to Advisory 3.0.3? If so, is there anything I can do to fix it?
(7) Do you know if this advisory could explain why the problem is intermittent for us? That is, it occurs on some units (usually repeatable) but not on others.
Thanks again!
You are welcome Doug,
If you don't find a resource conflict with ARM McASP driver and DSP side, you may want to dig into the SDMA/MDMA lockup issue in Advisory 3.0.3.
The problem has to do with a EDMA hardware deadlock situation that arises when the same EDMA TC is used to perform writes to BOTH DSP SDMA (L1, L2, and HDVICP RAM/Buffers through SDMA port) AND slave memories (DDR2, EMIFA, HDVICP0/1 EDMA ports, or ARM TCM)
Since EDMA is a shared peripheral, this may be caused by parallel accesses from multiple cores. For example, if McASP driver is using TC0 for issuing EDMA writes to one of the slave memories while simulatenously the codec on the DSP is using TC0 for writes to L2 the deadlock condition can arise.
The same condition can arise if a single DSP side codec, for example uses the same TC for writes to L2 and DDR simulatenously as well, but most codecs that are aware of the issue has probably addressed this by selecting different TCs for submitting transfers to based on the destination memory type. If you are able run the codecs without issue (until you start the ARM McASP) then chances are this is already taken care of. Otherwise you will need to contact your codec vendor. (I am sorry but I am not sure who that is, and you should go to the source).
One experiment you can do is to change the ARM or DSP side DMA channel to TC mappings. You can look at the EDMA DMAQNUMi registers (vIa CCS by displaying the contents of these registers - look up EDMA documentation for your board for the address and how to parse the register bits) There are 8 32-bit DMAQNUMi registers (i: 0..7) where each register encodes mappings for of 8 of the EDMA channels, and each channel is thus mapped to its associated TC Queue using the 4-bit encoding. All 64 channels are mapped this way. Observe what these registers are after linux side boot, to see what the kernel mapped. (Each driver potentially will set the entry for its own channel, so you may want to pause after the McASP driver is intalled and see the assignment. )
See if this is helpful,
Murat
Murat,
I read through the erratum more carefully yesterday and concluded that since I do not know what the codec is doing, it is indeed possible that this is an issue.
It looks like the deadlock can happen for memory transactions initiated by a source other than just EDMA TCs. All that matters is that the source generate a write command through the DSP's SDMA followed immediately by a memory write transfer to one of the other memory devices. Then under certain conditions deadlock ensues. On our board the sources can be the ARM data cache, an EDMA TC, PCI bus or VLYNQ(?). We use none of the others that are mentioned in the advisory.
One of the areas where I'm confused is the DSP SDMA. In the advisory it seems like this is a separate device, but in my reading of the interconnect diagram in spraaw4.pdf (DM6467 Architecture and Throughput), it appears that the DSP SDMA is really part of the caching infrastructure for the entire chip (both the ARM and DSP). Does that sound right to you?
I will take a look at how the TC's are configured and see. I know that our audio driver uses TI's EDMA services in the ARM Linux kernel to copy from McASP0 to DDR2. The reason I do not suspect our audio driver (as much!) is because we have two video input processors (what we call "channels") that are identical except that only one of them handles audio. Yet both channels appear to suffer from this problem (one at a time - that is, one channel's DM6467 will hang and the other's will not).
The codec is provided by TI. It is h264fhdvenc 01.10.02.03 (in DVSDK 3_10_00_19 GA). TI is generally very good about documenting their software - especially about the version numbers and what the test platforms were. But I saw nothing in the User's Guide (SPRUGN8D.pdf) about the advisory, nor did I see anything about whom to contact for support issues such as this. To me, the customer, it seems like TI is the point of contact for this codec.
(Q8) So could you please point me to someone who knows the source of the codec? Someone at TI must know.
If you don't mind, I will copy your message and my reply to the other thread (http://e2e.ti.com/support/embedded/f/354/t/140023.aspx "ARM hangs after Comm_create during instantiation of DSP codec"). We're really discussing that problem rather than "which DMA channels are used by ARM/DSP".
Thanks for all your help, Murat.
Doug
Hi Doug,
Yes, the issue can occur with several different scenarios. Since it is a race condition, it is still non-deterministic in nature, so where there is heavy concurrent DMA traffic into two submitted using the same TC master to say L2 and DDR2 we observe it often.
You are correct that you don't directly use SDMA. This is used internally by the cache controller, so indirectly SDMA could also be a culprit. Indeed one of the recommended workarounds in the errata is to "configure L2RAM as 100% cache to avoid stalls and deadlocks". This would have the affect of EDMA not being able to directly write to L2, since L2 type memory no longer exists (all of it utilized as cache). However I am not sure if you can use this workaround, as it will likely open a whole different can of worms: when your codecs can no longer receive the internal memory they ask for - both performance-wise, as well as cache coherence issues related to DMA usage.
I will try to alert someone from the codec team to pay attention to this thread and check if this particular codec has the right workarounds re: errata 3.0.3.
Also feel free to use any of the post to contribute to other discussion threads.
Best regards,
Murat
Murat,
Thanks. I would really appreciate a look by one of the codec gurus.
During an encoding run, I dumped the DMA channels being used by the codec and it appears that almost all access to L2 RAM is through TC2, and that no other destinations are used with TC2. The latter is definitely true. I never saw an instance of TC2 being used for anything other than L2 RAM.
Here's a quick list of what I've found:
TC | DMA Channels | Destinations |
TC0 | 0, 3, 14, 15, many above 20 | 404xxxxx (?), 11823530 (L2 RAM) |
TC1 | 13, many above 20 | 8Bxxxxxx (DDR2), 4060xxxx (?) |
TC2 | 2, 20, many above 20 | 1181xxxx, 1182xxxx (L2 RAM) |
TC3 | 1, 4, a few above 20 | 404xxxxx, 406xxxxx (?) |
But I did find that one channel (33) made had a single transfer from DDR2 to L2 RAM through TC0 when all the rest TC0's transfers were to 404xxxxx. Do you know what that region is? It is marked as "Reserved" in SPRS403G.pdf, page 22, but it is broken out separately from 40xxxxxx as its own 256K subsection. It's in the space where we'd expect to see EMIF CS1 Data, but that's not supported on DM6467/T. Is that region part of the HDVICP RAM/Buffers mentioned in the advisory? If so then I'm stuck. There's nothing I can do about it since it is all in the codec.
FYI, I found that my audio driver was also using TC0 for transfers to DDR2 from a McASP data buffer register. But that occurs only on one of our two processors. In fact, when I installed the kernel with it changed to TC1, it worked! I tried multiple video captures and they worked. I rebooted the ARMs and it still worked. Then I restored the original kernel and it still worked. Bummer. For whatever reason, the problem had disappeared completely. When I switched to another device that was failing on the other processor (the one that doesn't use the audio driver), it failed. Of course. While I might be able to remove a potential problem by changing the TC used by my audio driver, it is not the source of the problem in general.
Note that when I say "the codec used..." I really mean "either the Codec Engine, Codec Server or codec used..." I don't know which of the three software components allocated the DMA. I'm pretty sure that dsplinkk allocated the earlier channels.
The next possible culprit is the SPI driver. It uses DMA via TC3 for both receive and transmit. That means destinations both to its internal buffer register and to DDR2. I don't see how this could be the problem, though, unless conflicts with 40xxxxxx can cause the deadlock to occur.
So I leave you again with two requests:
(Q10) I would love to see comments from the codec team about all this.
(Q11) What are memory regions 404xxxxx and 406xxxxx in the DM6467/T? They're clearly used by the codec and support software (dsplinkk? - I'll investigate that).
Thanks for your posts, Murat.
Doug
Nice work Doug,
I am not sure if I have anything valuable to add to your latest post, asked a few pros to take a peek at this thread, though.
My hunch is that the codec is behaving fine, using TC2 exclusively for L2, and TC1 for DDR2. The TC0 L2 access may be an issue, depending on timing, and not sure who got assigned channel-33. There is an ARM side Timer0 IRQ tied to 33, but I am not sure if this is used to drive any DMA transfers, and not sure why it would generate L2 writes - just some questions, in case these ring bell.
I didn't quite understand the part about audio driver TC0->TC1 change working: do you mean that when you do this change you no longer observe the hanging when encoder runs?
Also, note that TC0 would service DMA requests at a higher priority than TC1 (unless QUETCMAP/QUEPRI changes were made to alter traffic/priorities) and you would want to run audio driver at highest priority for real-time purposes. Making this change might also be changing the timing of DMA events and masking issue that were leading to the hang, just more thoughts.
Cheers,
Murat
Murat,
Thank you for asking for a few pros to take a peek.
Yes, it looks like the codec is attempting to behave properly. Its use of TC2 exclusively for the purpose of transfers to L2 RAM makes me think so.
Yet... I wonder about the activities on TC0. That particular access to L2 RAM was on one of the DMA channels being used by the DSP (or the Codec Server/Engine) and NOT by any of our code. Clearly it is writing to L2 RAM and the mysterious 40xxxxxx address space.
About the audio driver.... No, the change from TC0 -> TC1 did not actually make the problem go away (and I know about the priority - and that is indeed why we had it in TC0). It was just coincidence that that particular unit ceased to exhibit the problem. I haven't seen it exhibit the problem since, even after restoring the original audio driver. Very disappointing, though I knew it could not be the solution - the audio driver is used on only one of the channels, yet both can suffer from the problem.
We just had a meeting today about this issue. It is beginning to hurt our business. We have a few unhappy customers. Also, I've been forced to try to solve this problem rather than continue further development for our product - development that is due to be released in two months. We're a small company, but through some of our partners/contacts are starting to pull some strings to get more timely and expert input about this issue from TI. You've been great at replying each day, but we're really starting to feel the pinch; time is of the essence.
If you can't get someone to look at this soon, then... do you know of anyone who would be willing to look at this as a consultant whom you believe to be competent enough to help us resolve it? We're willing to pay in beer, food or dollars!
Anyone else out there want to take a crack at it?
I'd like to re-iterate my last query since you didn't mention it:
(Q11) What are memory regions 404xxxxx and 406xxxxx in the DM6467/T?
It may not be in your area of expertise, but if you know someone who can answer, I'd appreciate it.
Thanks again, Murat.
Doug
Hi Doug,
I hope we can find something that alleviates the pain! I thought of a one more thing to explore.
Since you suspect channel-33 activity is coming from the DSP side, lets focus on that and see if DMAQNUM (channel to TC Q maps) could be causing an issue.
In the past I have seen a race condition involving DSP and ARM side drivers overstepping each other, causing channel-TC map to get corrupted. The issue has to do with the fact that each one of these 32-bit registers control the corresponding 8 EDMA channels TC mapping. Codecs that work around the 3.0.3 issue typically initialize the channel maps for the channels they are assigned to route traffic to the desired TC. However since the read-modify-write of the 32 bit register affects all 8 channels controlled by DMAQNUM[i], this needs to be an atomic read-modify-write. Problem is that, there is no hardware support to achieve this in a multi-core scenario. An ARM side driver updating DMAQNUM[i], to change the mapping of a channel k, where k is in {8*i, 8*i +1, 8*i +2, ... ,8*i+7} at the same time as another parallel update involving DMAQNUM[i] by the DSP side to update channel p, also p in {8*i, 8*i +1, 8*i +2, ... ,8*i+7} can lead to a race condition. For example, following atomic operations can take place in lock-step
ARM reads DMAQNUM[i] ;
DSP reads DMAQNUM[i] ;
ARM modifies 4-bits for 'chan-k' in DMAQNUM[i]
DSP modifies 4-bits for 'chan-p' in DMAQNUM[i]
DSP writes back DMAQNUM[i];
ARM writes back DMAQNUM[i];
Here, ARM write to DMAQNUM effectively overwrites the DSP's modification for chan-p, if the 4--bit value for 'p' was different when ARM read DMAQNUM in (1) (of course the opposite is also possible, but less critical, since chances are no-one else re-uses the ARM side's channel, whereas on the DSP same channel, p, could be used by different codecs, each mapping it to different TC during activation/deactivation.
In short, it might be interesting to experiment with excluding channels 33-40 (all 8 sharing the same DMAQNUM[4]) from DSP use. Alternatively, you may check the audio driver code to see (actually all drivers) if they repeatedly update DMAQNUM at truntime, as opposed to only during initialization, and shift this code to initialization to avoid potential for scenario above.
You may also want to dump the contents of DMAQNUM registers at runtime and observe if mappings are being altered before and after processing calls.
See if this helps,
Murat
Murat,
That's a good suggestion. Even though I usually know intuitively that bit-masked structures such as DMAQNUMx are thread-unsafe, for some reason I had not considered it in this situation. Ideally the Codec Server would negotiate these with the Codec Engine on the ARM. That would be the smart way to do it, though it would be at the expense of a dependency between the two sides.
I have looked at the DMAQNUMx values both before and after running the codec in the success case and in the fail case. Without knowing exactly what the codec is trying to do, I cannot make much of the list, except to say that our audio driver is getting the proper TC.
To avoid a race condition I will need to distribute DMA channels in groups of eight so that each core has exclusive access to a particular DMAQNUMx register. So the ARM might get 0-15 (DMAQNUM0-1) and the DSP 16-63 (DMAQNUM2-7). If I use the Codec Server as-is, the DSP claims channels in all 8 groups - even the channels for events marked as "Reserved" in Table 7-15 of the DM6467T User's Guide (SPRS605B.pdf page 163). I will definitely need to change the EDMA3_LLD allocations to be used by the DSP since it seems to be claiming all the DMAQNUMx registers.
I've noticed that the ARM likes to use the DMA channel numbers associated with its hardware events. For example, it will use channel 5 (AXEVTO0) for our audio driver since we're transferring two samples at a time (odd events only). Similarly, we use SPI (16-17), UART0 (18-19) and I2C (28-29). I don't think the UART driver uses DMA, and I see that there are no DMA resources being requested for I2C in arch/arm/mach-davinci/devices.c. Plus, I've never seen those DMA channels in use for device-specific things. But it looks like I need to reserve (on the ARM) DMA channels 0-7 and then a block of at least eight more somewhere else as general resources. Either that or I have to change the way channels and events are assigned (I think) in edma_alloc_channel() and affiliated services on the ARM side.
My main concern is that I will make the codec inoperable if I claim channels 0-7 and 16-23 for the ARM.
I will see about changing the structures in the Codec Server by modifying edma3_dm646x_cfg.c in the EDMA3_LLD library. That will take me a long time to set up. I'm running Win7/64 and Linux. My foggy memory tells me that I could not get the Codec Server to compile under CCS 4.0 after trying to import the projects. And I've recently re-tried to install CCS 3.3 under Win7/64, but to no avail. Any help there would be appreciated. I do remember getting it to work long ago, but the emulator drivers did not work for our XDS560(?) from Spectrum Digital, even with 3.3 R12.1 or later (it worked under XP). I would love to have the compiler available under Win7/64. We do have machines around here that are set up for it, so I'm not stuck; I'd just rather have it running on my own Win7/64 box.
Our chip distributor reps (from Arrow) just happened to stop by yesterday morning and they mentioned CCS 5.0 and an upcoming 5.1 release. That sounded promising for future development on newer chips, but I suspect it may not be suited for our DSP. Do you know anything about that? Should I bother trying to compile EDMA3_LLD using that?
Thanks for the insight on DMAQNUMx. It seems like an unlikely culprit since modifying DMAQNUMx occurs so seldom on the ARM side, but it's definitely a potential problem that I will try to fix.
Doug
Murat,
A major question which I may have forgotten is this:
(Q12) Given that you know customers have had this problem (both Advisory 3.0.3 and the DMA/DMAQNUMx resource conflict), how did they solve the problem?
Thanks,
Doug
Two things:
1. ARM side drivers updated to only initialize DMAQNUM entry during initial creation. Some drivers were performing this with high frequency during runtime, increasing the chance of contention on the entry with remote cores. You might want to review the audio driver code to see where this is being done.
2. partitioning EDMA channels in groups of 8 (aligned on DMAQNUM boundaries.) and allocating them statically to each core in this way.
Note that, this has been observed more frequently on multi-core devices (Shannon/NyQ) and so most ARM+DSP environments are probably not seeing the issue very much.
Also, a question: when you dump DMAQNUM's what do you see for channel-33's mapping? That might hold some clues. If you do not see it mapped to TC0, (which is what I would expect for the DSP side usage) then the mystery L2 transfer on TC0 from channel-33 should be investigated, and DMAQNUM race-condition is certainly a possible cause.
Murat
Murat,
Thanks for the speedy reply.
I checked the audio driver and the SPI driver earlier and both call edma_alloc_channel() (which calls map_dmach_queue() to set DMAQNUMx) at probe time. The SPI driver is built into the kernel. The audio driver is loaded by a boot script. So only the DSP is allocating these at run time. So in retrospect this should have been foreseen as unlikely.
Here are the DMAQNUMx values for each channel before and after a capture (channel 1 is on the left; channel 2 is on the right):
echo085976 cap # lua -lgb -e "for i=0,7 do px(dm1.edma.DMAQNUM[i](),dm2.edma.DMAQNUM[i]()) end"
11131111 11031111
11111111 11111101
11111133 11111133
11111111 11111111
11111111 11111111
11111111 11111111
11111111 11111111
11111111 11111111
echo085976 cap # ./cap.lua ch1 archive 1k ch2 archive 1k 10
... capture fails ...
echo085976 cap # lua -lgb -e "for i=0,7 do px(dm1.edma.DMAQNUM[i](),dm2.edma.DMAQNUM[i]()) end"
11130230 11030230
00111111 00111101
01021133 01021133
12003020 12003020
20000103 20000103
00322312 00322312
11222200 11222200
23321123 23321123
Too bad I can't reduce the inter-line spacing for that. I tried pasting from Word, too. No luck.
In the case above, the capture started and got about 8 seconds in before it failed. Usually it dies before that - like either immediately or less than a second into the encoding.
Notice that the 5th word in both columns ends in 03. The 0 is for TC0 on DMA channel 33. I did not dump the channel parameter mappings on this run so I cannot say that they are the same.
Sometimes the DSP does not get to the point of running the codec before the DM6467 locks up, so then there are no changes to DMAQNUMx. But it ALWAYS gets to the point of trying to start the DSP. My hardware guys say that the power rails are fine. That's another possibility. I'll have them re-check that. For what it's worth, there's some sort of auto-reset on the voltage regulator and it never gets triggered.
Murat,
I'm sorry, but my Q12 wasn't clear enough. I meant to ask:
(Q12) How did your other customers resolve problems with DM6467 Advisory 3.0.3 (rather than the race condition in DMAQNUMx)?
And do you know about memory regions 404xxxxx and 406xxxxx? They're heavily used by the DSP. Mostly I need to know if those regions are HDVICP buffers, or if they're considered like DDR2 - they must not be in the same TC as one being used for L2 RAM.
Thanks again for you prompt attention,
Doug
Hi Doug,
I am probing into the memory regions 404xxxxx and 406xxxxx, and I do suspect these are unpublished HDVICP memory references. We will find out.
Working on the HDVICP assumption, the 3.0.3 advisory applies to HDVICP memory targets as well. So you should not use the same TC for simultaneous writes to L2 and HDVICP (similar to DDR2).
Looking at your DMAQNUM findings, showing that channel-33 was mapped to TC0 could be causing a 3.0.3 lockup, so I would still highlight that.
Regarding your question (Q12)How did your other customers resolve problems with DM6467 Advisory 3.0.3 (rather than the race condition in DMAQNUMx)?
we have introduced enhancements in Framework Components for codecs to have some control over the TC selection for separating DMA transfers into internal & external memory spaces. Codecs with the workarounds use these enhancements. To give you an idea as to what happens there:
For codecs that use IRES/RMAN to acquire EDMA resources, we have configuration support via "ti.sdo.fc.edma3.Settings" package, where it is possible to select the TCs to use for transfers to internal vs external.
EDMAYou cannot directly assign which TC to be used, instead you would do this using the ECPY & IRES_EDMA3CHAN supported abstraction, where you assign the actual TCs via configuration. Codecs which use IRES_EDMA3CHAN and ECPY then call ECPY_createHandle to create distinct DMA handles for wrinting to internal and external/hdvicp memories. Say, the following handles: ehandle_int & ehandle_ext
Then to map these two channels to two different TC Queues, one for internal the other for external memory destination type transfers codec which uses ECPY calls:
ECPY_setDestinationMemoryMode(ehandle_int, INTMEMORY0);
ECPY_setDestinationMemoryMode(ehandle_ext, EXTMEMORY0);
Subsequent calls to ECPY uses the information obtained from the resource manager to select the actually TC Queue, which was configured via ti.sdo.fc.edma3.Settings. The mappings exist within each IRES_EDMA3CHAN handle ( via: handle->iresHandle->queueMap[], where queueMap indexes as the same as the IRES_EDMA3CHAN_DmaDestType), so codecs that don't use ECPY can also get the TC assignments from the handle directly.
Just to give you an idea. So it should be the codec's responsibility to sort out transfers and use configured TCs for the different transfer types. I suspect your codecs are already doing this.
Best regards,
Murat
Hi Doug,
I got some feedback from the Codec experts, who confirmed that:
1. This third-party codec actually does engage the workaround for Errata 3.0.3, and submits all L2 writes on the dedicated TC2
2. The 0x404xxxxx and 0x406xxxxx adress ranges are indeed in the reserved HDVICP memory space.
So, all EDMA writes to this region should not be done on any TC which writes to L2. Based on your observation #1:
TC | DMA Channels | Destinations |
TC0 | 0, 3, 14, 15, many above 20 | 404xxxxx (?), 11823530 (L2 RAM) |
TC1 | 13, many above 20 | 8Bxxxxxx (DDR2), 4060xxxx (?) |
TC2 | 2, 20, many above 20 | 1181xxxx, 1182xxxx (L2 RAM) |
TC3 | 1, 4, a few above 20 | 404xxxxx, 406xxxxx (?) |
and your observation #2: that EDMA channel-33 owned by the DSP issued an EDMA transfer into HDVICP memory via TC0 can easily be the reason you get a lock-up while the ARM is submitting writes to L2 via TC0 simultaneously.
At this point you should try to exclude channels 32..39 from allocation to the DSP and see if that resolves your issue.
Best regards,
Murat
Murat,
Excellent! Thank you for tracking down those answers. That is very helpful.
I wonder why TI did not just say "Reserved (HDVICP internal RAM)" in the data sheet? That would have helped. Or at least the erratum could had mentioned the memory location of HDVICP RAM/Buffers since it is such an important (potential) part of the problem.
This definitely gives me something to look at.
FYI, I have CCS 3.3 installed with the simulator on my Win7/64 box. I have not yet opened the EDMA3_LLD project files to see about building it. I hope it's not too much trouble, otherwise I might try hacking the binary to change the DMA allocations. That's a serious hack, but it has the advantage of being quick!
Thanks again,
Doug
Murat,
I have verified that all my ARM DMA channel allocations occur before the DSP is started, so I do not expect any race conditions with the DSP over DMAQNUMx register contents.
So I did not remap the DMA in the Codec Server. That will end up being too big of a job; it should have been done by TI. Instead, I programmed the ARM to avoid any channels that are claimed by the DMA (as mentioned very early in this thread).
Yesterday I found that the older h264venc (720p) encoder avoided the problem. I have taken a unit that would nearly always fail with the h264fhdvenc (1080p) encoder and run it for hours capture hundreds of times, with many ARM reboots sprinkled in there, and it never failed.
This morning an executive at my company pointed me to this link:
http://e2e.ti.com/support/embedded/f/354/p/102526/405884.aspx
I don't know how I missed it in my searching. There have been no meaningful replies there, even to a TI rep's request.
Are you sure that the codec guys said there are no problems with the 1080p encoder, h264fhdvenc? Are you sure they were referring to the proper codec?
We need the 1080p encoder. This is not good. TI has some slick stuff, but if it makes us look bad to our customers, we'll have to find other solutions.
I hope we can get a more thorough reply from someone who is an expert in the source code for these codecs.
Thanks for the help you've given so far.
Doug
Murat,
Just FYI, I found that the 720p encoder uses TC0 (on DMA channel 30) to write to L2 RAM. As you may recall, the 1080p encoder was using TC0 via DMA channel 33.
Interestingly, the 720p encoder does not use TC2 at all. So is the 1080p encoder just the 720p encoder, but using EDMA3 and L2 RAM for a performance boost in certain places? That seems like a good guess.
The fact that they both use TC0 for access to L2 RAM (on slightly different DMA channels) makes me think that that channel (30 or 33) is being used by an infrastructure component and not by the codec itself. It's not being used by the ARM because I've marked them all as reserved in the kernel. But if it is an infrastructure component, then at least I should have the source code. I could spend a few more days dumpster-diving for the offending code. Or maybe someone at TI who actually knows the code can do that - find where the Codec Server is using TC0 and change it to TC2.
As long as TC0 is used by the DSP to write to L2 RAM, I will never be safe using TC0 with my audio driver to write to DDR2. If I choose to map it to another TC, I cannot choose TC2 if I want to avoid conflict with the 1080p encoder. That leaves TC1 or TC3. At the default priorities, TC1 would be a better choice to ensure that I don't lose audio samples. Or I could switch the priorities, which would allow a bunch of other potential priority inversions in the codec.
Regardless, the problem may still occur. The audio driver is on only one of our channels, and both have been seen to lock up.
Do you have any other ideas?
Doug
Murat,
One final posting for the night (at least it's night for me!).
Your last posting had an error in it. You said that the ARM was writing to L2 RAM through TC0. That is not true. It is the DSP that allocated the channel (33) that writes to L2 RAM via TC0.
Doug
Murat, Girish and Kapil,
Here's what I get for the DMAQNUM registers after running mapdmaq-hd:
root@ghostbuster /root# /sbin/mapdmaq-hd
root@ghostbuster /root# /home/drogers/svn/4-0-x/gbfirmware/exes/capture_arm/tools/read_edma_regs
Mapped 0x00000188 bytes at 0x01C00100 to virtual address 0x40020100.
Mapped 0x00004000 bytes at 0x01C04000 to virtual address 0x40193000.
DMAQNUM0 = 11132222
DMAQNUM1 = 22211111
DMAQNUM2 = 31311133
DMAQNUM3 = 22332222
DMAQNUM4 = 11111111
DMAQNUM5 = 22222111
DMAQNUM6 = 11111111
DMAQNUM7 = 22222222
QDMAQNUM = 00000000
QUEPRI = 00001533
CHAN TC# PARAM___ @PhysAddr: ___SRC__ ___DST__ ___OPT__ ACNT BCNT SBIX DBIX BCTL CCNT SCIX DCIX
root@ghostbuster /root#
Hi Doug,
There are multiple issues, some confirmed others under investigation.
Confirmed:
#1 1080p encoder issuing HDVICP writes via TC0, which is clearly happening on your traces, as well as confirmed by the codec team.
Investigated:
#2 VDCE driver submitting writes to DDR via TC2?
Clearly, this would issue problems since TC2 is to be used for L2 writes.
#3 Other writes to HDVICP via TC2 (presumably codec, using channels: 42,44,49) as shown on your trace.
I am suspecting this could be due to DMAQNUM corruption.
You should be investigating #3 on your end. As per my earlier posts on this thread.
#1 and #2 are issues that should be corrected by codec and driver vendor.
Cheers,
Murat
This is definitely problematic, considering we are also seeing TC0 used for L2 writes by ARM drivers (via channel-33, ...)
Murat,
For some reason I did not see your post until today.
Let me address the very last comment - channel 33 is NOT being used by the ARM (at least, not by anything through the Linux kernel edma3 service). I have explicitly set it aside as noted earlier in the post. Here's the entry in arch/arm/mach-davinci/dm646x.c:
static const s8 dm646x_dma_rsv_chans[][2] = {
/* (offset, length) */
{ 0, 4},
{13, 3}, /* Available before: 5-12 */
{24, 8}, /* Available before: 16-23 */
{32, 22},
{56, 8}, /* Available before: 54-55 */
{-1, -1}
};
Also, I don't suspect DMAQNUM corruption as much as you do - at least not corruption due to a race condition between the processors. My tests show the same results each time, which makes me wary of attributing the queue allocations to such a race condition.
However, I did notice that there's a (kind of) bug in the handling of the DMA queue. One of the macros, EDMA_CHAN_SLOT() in arch/arm/mach-davinci/include/mach/edma.h as used by drivers/char/davinci_vdce.c, uses a 16-bit mask, 0x0000FFFF, to mask off a 6-bit event number. That value is then it shifted up 12 bits by EDMA_CC() into position and simply OR'd it into the param set's OPT entry - without first clearing the space for it (in that function, edma3_memcpy()). I found that it was cleared elsewhere upon initialization and (I think, from dumping the param sets) that the values are cleared when they are no longer used. So ultimately I think the code is a bit untidy, but the overall effect should be correct.
As for the VDCE's event queue, I believe it is using TC3, not TC2. In fact, I believe we're seeing artifacts in the chroma conversion because it is in TC3; it explains one of our problems which we do not see in the absence of the codec. I may be moving it to TC1 in the near future.
Thanks for the comments. I definitely appreciate all the feedback.
Doug
From Echo360 yesterday, request for TI to examine code - Customer says this should provide clear evidence of the problem to you.
I had sent this in private correspondence with Murat. The program is very short - it never calls Venc1_process(), just Venc1_create().
Here's the essence of the program:
eng = Engine_open((Char*) engine_name, NULL, NULL);
printf(PROGRAM ": Engine_open() returned.\n");
if (NULL == eng)
{
fprintf(stderr, PROGRAM ": Engine_open(\"%s\") returned NULL.\n", engine_name);
goto CLEANUP;
}
printf(PROGRAM ": Engine_open(\"%s\") succeeded.\n", engine_name);
printf(PROGRAM ": calling Venc1_create(\"%s\")...\n", codec_name);
enc = Venc1_create(eng, (Char*) codec_name, params, dyn_params);
if (NULL == enc)
{
fprintf(stderr, PROGRAM ": Venc1_create(\"%s\") failed.\n", codec_name);
goto CLEANUP;
}
printf(PROGRAM ": Venc1_create(\"%s\") succeeded.\n", codec_name);
result = 1;
CLEANUP:
if (NULL != enc) Venc1_delete(enc);
if (NULL != eng) Engine_close(eng);
return 0;
Here's the email about it:
Murat,
I've attached simple_encode.c (though it does no encoding). This works when
I pass h264enc on the command line. The output is:
root@ghostbuster bin# /home/default/simple_encode h264enc
simple_encode: open_engine_codec() enter.
simple_encode: calling Engine_open("ezenc")...
simple_encode: Engine_open() returned.
simple_encode: Engine_open("ezenc") succeeded.
simple_encode: calling Venc1_create("h264enc")...
simple_encode: Venc1_create("h264enc") succeeded.
root@ghostbuster bin#
When I pass h264fhdvenc (or don't pass anything - see the attached code) and
run with CE_DEBUG=3, I get a huge stream of messages (also attached) that ends with:
[DSP] @0,113,873tk: [+0 T:0x8ba0006c S:0x8ba0402c] CR - processRmsCmd(0x8ff03928, 4056): cmd = 5
[DSP] @0,113,928tk: [+0 T:0x8ba0006c S:0x8ba0402c] CR - remote time = 0x0, trace buffer size = 4032
@1,377,534us: [+0 T:0x4001e970 S:0xbe93bb4c] CE - Engine_fwriteTrace> returning count [15656]
@1,377,659us: [+0 T:0x4001e970 S:0xbe93bbb4] CE - Engine_open> return(333008)
simple_encode: Engine_open() returned.
simple_encode: Engine_open("ezenc") succeeded.
simple_encode: calling Venc1_create("h264fhdvenc")...
@1,380,896us: [+2 T:0x4001e970 S:0xbe93bb3c] ti.sdo.dmai - [Venc1] Creating encoder h264fhdvenc for max 1920x1088 bitrate 20000000 ratectrl 4
@1,381,127us: [+0 T:0x4001e970 S:0xbe93bb14] ti.sdo.ce.video1.VIDENC1 - VIDENC1_create> Enter (engine=0x514d0, name='h264fhdvenc', params=0x4e554)
@1,381,298us: [+0 T:0x4001e970 S:0xbe93bae4] CV - VISA_create(0x514d0, 'h264fhdvenc', 0x4e554, 0x520, 'ti.sdo.ce.video1.IVIDENC1')
@1,381,441us: [+0 T:0x4001e970 S:0xbe93b9dc] CV - VISA_create2(0x514d0, 'h264fhdvenc', 0x4e554, 0x30, 0x520, 'ti.sdo.ce.video1.IVIDENC1')
@1,381,594us: [+0 T:0x4001e970 S:0xbe93b9c4] OM - Memory_alloc> Enter(0x34)
@1,381,726us: [+0 T:0x4001e970 S:0xbe93b9c4] OM - Memory_alloc> return (0x51908)
@1,381,848us: [+0 T:0x4001e970 S:0xbe93b9c4] OM - Memory_alloc> Enter(0x4)
@1,381,972us: [+0 T:0x4001e970 S:0xbe93b9c4] OM - Memory_alloc> return (0x51940)
@1,382,091us: [+0 T:0x4001e970 S:0xbe93b9bc] OC - Comm_alloc> Enter(poolId=0x0, msg=0x51940, size=520)
@1,382,259us: [+0 T:0x4001e970 S:0xbe93b9bc] OC - Comm_alloc> msg=0x41216900, returning (0)
@1,382,389us: [+0 T:0x4001e970 S:0xbe93b964] CE - Engine_createNode(0x514d0, 'h264fhdvenc', 520, 0x4e554, 0x30, 0xbe93bad8)
@1,382,522us: [+0 T:0x4001e970 S:0xbe93b964] CE - Engine> allocNode Enter(engine=0x514d0, impId='h264fhdvenc')
@1,382,643us: [+0 T:0x4001e970 S:0xbe93b94c] OM - Memory_alloc> Enter(0x20)
@1,382,764us: [+0 T:0x4001e970 S:0xbe93b94c] OM - Memory_alloc> return (0x51760)
@1,382,899us: [+0 T:0x4001e970 S:0xbe93b964] CE - Engine> allocNode(). Calling (Comm_create(gppfromnode_2012_1, 0x51768, NULL)
@1,383,028us: [+0 T:0x4001e970 S:0xbe93b93c] OC - Comm_create> Enter(queueName='gppfromnode_2012_1', queue=0x51768, attrs=0x0)
@1,383,158us: [+0 T:0x4001e970 S:0xbe93b924] OM - Memory_alloc> Enter(0x4)
@1,383,280us: [+0 T:0x4001e970 S:0xbe93b924] OM - Memory_alloc> return (0x51950)
@1,384,541us: [+0 T:0x4001e970 S:0xbe93b93c] OC - Comm_create> return (0x51950)
@1,384,714us: [+0 T:0x4001e970 S:0xbe93b944] OC - Comm_put> Enter(queue=0x0, msg=0x41215900)
@1,384,905us: [+0 T:0x4001e970 S:0xbe93b944] OC - Comm_put> return (0)
@1,385,042us: [+0 T:0x4001e970 S:0xbe93b93c] OC - Comm_get> Enter(queue=0x10000, msg=0xbe93b9ec, timeout=-1)
And that's when it hangs. This is what I posted in the other thread at the
forum (http://e2e.ti.com/support/dsp/davinci_digital_media_processors/f/99/p/140023/523321.aspx).
Any ideas?
One thing I noticed is that the directory structure being reported by the codec
server (/home/drogers/newclient/vendor/ti/dvsdk/....) has not been used in a very
long time. I believe I did install it there originally. Since then it has been
moved. The directory being reported by the codec engine is correct
(/home/drogers/svn/4-0-x/gbsdk/...). I would be surprised if that's an issue, but
I wanted to point it out.
Let me know what you think.
Doug
Hi Doug,
I'm so glad to see your post. My platform is also DM6467T. I want to known how to look for EDMA resources when the program is runing and how to configure EDMA resources in codec server. Specially, for the TC DMA channels.
Where or how you you find that the information as follow?
TC | DMA Channels | Destinations |
TC0 | 0, 3, 14, 15, many above 20 | 404xxxxx (?), 11823530 (L2 RAM) |
TC1 | 13, many above 20 | 8Bxxxxxx (DDR2), 4060xxxx (?) |
TC2 | 2, 20, many above 20 | 1181xxxx, 1182xxxx (L2 RAM) |
TC3 | 1, 4, a few above 20 | 404xxxxx, 406xxxxx (?) |
The problem that I encount is when I domnload a pair mpeg4 encoder and decoder from TI official website, they can work only alone but deadlock at the same time.
I check'd the mem resource and edma channels in the server.cfg file, they should enough. Additonally, I see the note in the file mpeg4decoder datasheet ,
Resource requirement:
The entire HDVICP (0/1) is a video resource and uses 16K ITCM and 8K DTCM.
18 EDMA channels and 64 additional param entries are required.
TC-3 is used for transferring data from DDR to L2
I doubt that the deadlock relates about the "TC-3 is used for transferring data from DDR to L2 ". But my doubt isn't confirmed because I have not fund how to configure the TC DMA channels.
Hopefully, you can give me some suggest.
Regards,
LL