NDK PPP SEND VIA SPECIFIC INTERFACE

Hi Barry,

I did some experimenting with this using an app that also has 2 interfaces configured, although slightly different (IF 0 = Ethernet, IF 1 = PPP).

I was expecting that bind()-ing to a particular IF would do what you wanted, but it does not.  All bind does is force the source address of the UDP packet to have the IP of the interface that the socket was bound to in the bind() call.  The packet still goes out on the IF that makes sense based on the destination/server IP address.

It seems that the routing table is governing this behavior.

Can you print out your route table and paste that back to this thread?

If you are able to, given your set up, an easy way to do this is to telnet into your board from a PC.  Once you have telnet'ed in, you can display the routing table using the command "route print"

You can easily add a Telnet server to your app with the following 2 steps:

1. Add the following (global variable) code to one of your C files (it's required by the Telnet server):
   1. char *VerStr = "\nNDK Telnet Server\n";
2. Add the following code to your *.cfg file:

var Telnet = xdc.useModule("ti.ndk.config.Telnet");
var telnetParams = new Telnet.Params();
telnetParams.ifIdx = X; /* choose the IF you want to connect to with Telnet */
telnetParams.callBackFxn = '&ConsoleOpen';
var telnet = Telnet.create(telnetParams);

In any case, I think what you really are looking for is the socket option that's available in Linux called SO_BINDTODEVICE.  This option binds a socket to a given interface.  The NDK does not support this option as of today.

I've filed an enhancement request for this:

SDOCM00118246 Need to support socket option SO_BINDTODEVICE

Steve

Thanks for that, SO_BINDTODEVICE is exactly what I would like.

I added a route print from an example in the NDK (DumpRouteTable) so I didn't need to telnet into the 3G link.

Route table with two PPP sessions established is as follows:

  Address          Subnet Mask      Flags   Gateway                              
---------------  ---------------  ------  -----------------                    
0.0.0.0          0.0.0.0          U       if-2                                 
0.0.0.0          0.0.0.0          U       if-3                                 
10.106.107.194   255.255.255.255  U H  L  local (if-2)                         
10.217.92.164    255.255.255.255  U H  L  local (if-3)

Hi Barry,

My apologies for not responding, this one slipped through the cracks.

What's the destination IP address?  Based on your route table, your UDP packet's probably getting routed through the default route (IF-2 in this case).

I'm wondering if you can add a static route for the destination IP address you are trying to reach and then specify IF-3 as the outgoing interface.  Can you try this and see if it allows your UDP packet(s) to go out on IF-3?

The issue with this is that such a route entry will cause all packets destined for that IP address to go out on IF-3.  But if you can try this as a first step, I might have a (crude) idea that may allow you to work around this problem.

Steve

You can find example code of how to add a static route in the code for the Telnet command "route add". See ti/ndk/tools/console/conroute.c

Hello Steven

Thanks for the response.

I have looked at the example and can work out what I set the IP address and mask to

What do I set the gateway to ?

Is it my PPP IP address for the interface?

Regards

Barry

Barry,

I did some more experimenting this morning.  I think you actually should use a different method/API than what the Telent add route command does (sorry!).

I was able to do the following:

1. bring up my dual IF example with

• IF1:
  • Ethernet
  • IP address: 10.90.90.24
• IF2
  • PPP
  • IP address: 172.16.0.4
  • (directly connected via USB connection to my PC IP address 172.16.0.2)

2. At this point, without me doing anything explicitly, the route table is as follows:

>route print

Address Subnet Mask Flags Gateway
--------------- --------------- ------ -----------------
0.0.0.0 0.0.0.0 U if-2
10.90.90.0 255.255.255.0 U C if-1
10.90.90.1 255.255.255.255 U H X: X: X: X: X: X
10.90.90.24 255.255.255.255 U H L local (if-1)
172.16.0.4 255.255.255.255 U H L local (if-2)

3. send a UDP packet with destination IP address 172.16.0.2 -> results in UDP packet going to correct host over IF2 (via default route).  I see this on Wireshark for the PPP IF on my PC.

4. add a new static route for the host IP address 172.16.0.2, but specify IF1 (the Ethernet IF) - See code attached.

Address Subnet Mask Flags Gateway
--------------- --------------- ------ -----------------
0.0.0.0 0.0.0.0 U if-2
10.90.90.0 255.255.255.0 U C if-1
10.90.90.1 255.255.255.255 U H X: X: X: X: X: X
10.90.90.24 255.255.255.255 U H L local (if-1)
172.16.0.2 255.255.255.255 U HS
172.16.0.4 255.255.255.255 U H L local (if-2)

    (note the gateway for the new entry is blank, not sure why that is [perhaps an issue with the route print command] but the gateway is indeed IF1)

5. Repeat #3 above.  This time, I see the UDP packet go out on the Ethernet IF.

So, by adding a route entry for the specific IP address + interface, I was able to force the PPP packet to go out on the Ethernet IF.

Can you give the code that's attached a try?  Note you will need to modify the IP address values.  The code to add a route is under "if (command == 1)"

Steve

Edit (8/17/2015): removed previous attachment and replaced with the correct file below.  Note that in my test, createRoute() runs as a Task function.

1184.createRoute.c

Hello Steve

Thanks for your help on this.

I am very happy to give this a try.

However I downloaded the udpclient.c file and there is no line "if (command == 1)" the word command doesn't appear anywhere in the file?

Regards

Barry

Barry,

Great that you got it working!

I apologize, in the previous post I attached the wrong file.  I'll update the attachment in that post to have the correct file, in case others would like to see it if they come across this thread.

Steve

Thanks Steve, new file makes more sense.

Your routing table information pointed me in the right direction though as it showed me the flags you had set.

One thing I noticed is you don't use llenter() and llexit() around the rtcreate()

Should they be called around this?

Hello Steve

Over last few weeks I have noticed an issue with the static routes.

The create route works for me with the PPP interface no matter how long I run the application for.

However I am now finding that the Ethernet fails to create the route after I leave the application run for about 3 days.

Every time I go to send on any interface I first check if the static route to destination exists for that interface.

If it doesn't I create it, the Ethernet route shows up as having the MAC address.

Address          Subnet Mask      Flags   Gateway
---------------  ---------------  ------  -----------------
0.0.0.0          0.0.0.0          U       if-2
10.251.141.186   255.255.255.255  U H  L  local (if-2)
192.168.5.0      255.255.255.0    U   C   if-1
192.168.5.177    255.255.255.255  U H  L  local (if-1)
192.168.5.183    255.255.255.255  U HS    00:0B:AB:15:92:D3
203.59.8.178     255.255.255.255  U HS    if-2

After some time period the Ethernet route disappears.

This is not normally a problem as I simply create it again when I need it.

However eventually the route just isn't created any more?

Address          Subnet Mask      Flags   Gateway
---------------  ---------------  ------  -----------------
0.0.0.0          0.0.0.0          U       if-2
10.251.197.197   255.255.255.255  U H  L  local (if-2)
192.168.5.0      255.255.255.0    U   C   if-1
192.168.5.177    255.255.255.255  U H  L  local

Here is how I add the route

//=========================================================================
// AddStaticRoute()
//
// Adds a static gateway route to the system. The IP host or
// subnet accessible via the gateway is specified in IPTargetAddr
// and IPTargetMask. For a default route, both of these are NULL.
//
// Returns HANDLE to Route
//=========================================================================
HANDLE AddStaticRoute( IPN IPTarget, int IfIdx , uint8_t diagsOn)
{
  HANDLE 					hRt;
  NETIF_DEVICE *  ptr_device;

  // Don't add if we have existing route for this target IP address on this interface
	if (StaticRouteExists(IPTarget, IfIdx, diagsOn))
		return NULL;

	ptr_device = NIMUFindByIndex(IfIdx);

	if (diagsOn)
		LogDiagMessage("IF-%d:%d.%d.%d.%d Static route add %8x", IfIdx,
									 (UINT8)(IPTarget >> 0) & 0xFF, (UINT8)(IPTarget >> 8) & 0xFF,
									 (UINT8)(IPTarget >> 16) & 0xFF, (UINT8)(IPTarget >> 24) & 0xFF, ptr_device);

  // Start calling STACK functions
  llEnter();

  // Create the route and make it STATIC
  hRt = RtCreate( FLG_RTF_REPORT, FLG_RTE_HOST | FLG_RTE_STATIC, IPTarget, 0xFFFFFFFF, ptr_device, 0, 0 );

  // Since the route is STATIC, we can DeRef it here, and we
  // don't have to worry about keeping track of it.
  if( hRt )
  	RtDeRef( hRt );

  // Stop calling STACK functions
  llExit();

  // Return an indication of success
  return( hRt );
}

Here is how I check if it exists

//=========================================================================
// StaticRouteExists()
//
// Checks if a route already exists
//=========================================================================
int StaticRouteExists( IPN IPTarget , int IfIdx, uint8_t diagsOn)
{
	HANDLE 					hRt;
	HANDLE 					hIF;
	uint   					wFlags;
	UINT32 					IPAddr;
	NETIF_DEVICE *  ptr_device;
	int    					resp = 0;

	ptr_device = NIMUFindByIndex(IfIdx);

	// Start calling STACK functions
	llEnter();

	// Start walking the tree
	hRt = RtWalkBegin();

	// Search while there's more to search
	while (hRt)
	{
		IPAddr = RtGetIPAddr(hRt);
		wFlags = RtGetFlags(hRt);
		hIF    = RtGetIF(hRt);
		// The IP target must match and it must be STATIC HOST
		if((IPTarget == IPAddr) &&
			 ((wFlags & (FLG_RTE_STATIC | FLG_RTE_HOST)) == (FLG_RTE_STATIC | FLG_RTE_HOST)))
		{
			// Rule already exists on this interface
			if (hIF == ptr_device)
			{
				resp = 1;
			}
			else
			{
				if (diagsOn)
					LogDiagMessage("IF-%d:%d.%d.%d.%d Route removed from other IFACE", IfIdx,
												 (UINT8)(IPTarget >> 0) & 0xFF, (UINT8)(IPTarget >> 8) & 0xFF,
												 (UINT8)(IPTarget >> 16) & 0xFF, (UINT8)(IPTarget >> 24) & 0xFF);
				// Same Target is on different interface so remove it
				RtRemove( hRt, FLG_RTF_REPORT, RTC_NETUNREACH );
			}
			break;
		}
		hRt = RtWalkNext( hRt );
	}

	// Finish calling STACK functions
	RtWalkEnd( hRt );

	llExit();
	// Return with found count
	return( resp );
}

I am trying to work out why I can no longer add the static route?

Since the old ones are removed I wouldn't expect any sort of memory issue.

Would value any feedback you can offer

Further information on adding static route, adding no route to Ethernet on LAN worked fine.

However If I add a destination address which is not on the LAN (Internet address) as the static route for the Ethernet.

The route is changed from UP to down after a single packet attempt.

In the rtCReate, I have tried setting the gateway to IP address of our gateway, setting it to NULL or setting it to 0xFFFFFFFF

Regardless of the value I use the route only stays up for a single packet send attempt and then goes to down.

The packet never reaches the target address.

Setting the static IP on the PPP interface correctly delivers the packet and route stays up.

Eventually the adding of route fails and the sendto function returns a value of -1,

fdError() returns a value of 65 which indicates EHOSTUNREACH

Finally got this to work.  It's not elegant but it does work.

On Ethernet side - don't add any routes at all, you just need the gateway for WAN packets and route will match LAN addresses by default.

Adding routes can actually cause the routing to Ethernet to fail.

On the PPP side I add a route before doing the sendto and then remove the route immediately.

My sends are wrapped in GateMutex so both interfaces cannot send at the same time (This suits my needs but may not suit others, as I said above not elegant but works).

I created three functions to add and remove the static routes to direct packet to PPP

//=========================================================================
// RemoveStaticRoute()
// Removes any STATIC route belonging to the specified interface
//=========================================================================
int RemoveStaticRoute( int IfIdx, uint16_t diagsOn)
{
	HANDLE 					hRt;
	HANDLE 					hIF;
	UINT32 					IPAddr;
	uint   					wFlags;
	NETIF_DEVICE *  pNetIF;
	int    					resp = 0;

	pNetIF = NIMUFindByIndex(IfIdx);

	// Start calling STACK functions
	llEnter();

	// Start walking the tree
	hRt = RtWalkBegin();

	// Search while there's more to search
	while (hRt)
	{
		IPAddr = RtGetIPAddr(hRt);
		wFlags = RtGetFlags(hRt);
		hIF    = RtGetIF(hRt);
		// The IF must match and it must be STATIC HOST
		if((hIF == pNetIF) &&
			 ((wFlags & (FLG_RTE_STATIC | FLG_RTE_HOST)) == (FLG_RTE_STATIC | FLG_RTE_HOST)))
		{
			if (diagsOn)
				LogDiagMessage("IF-%d:%d.%d.%d.%d Route removed from IFACE", IfIdx,
											 (UINT8)(IPAddr >> 0) & 0xFF, (UINT8)(IPAddr >> 8) & 0xFF,
											 (UINT8)(IPAddr >> 16) & 0xFF, (UINT8)(IPAddr >> 24) & 0xFF);
				// Same Target is on different interface so remove it
				RtRemove( hRt, FLG_RTF_REPORT, RTC_NETUNREACH );
				resp++;
		}
		hRt = RtWalkNext( hRt );
	}

	// Finish calling STACK functions
	RtWalkEnd( hRt );

	llExit();

	// Return with found count
	return( resp );
}

and to add the route

//=========================================================================
// AddStaticRoute()
//
// Adds a static gateway route to the system. The IP host or
// subnet accessible via the gateway is specified in IPTargetAddr
// and IPTargetMask. For a default route, both of these are NULL.
//
// Returns HANDLE to Route
//=========================================================================
HANDLE AddStaticRoute( IPN IPTarget, IPN IPGateway, int IfIdx , uint16_t diagsOn)
{
  HANDLE 					hRt;
  NETIF_DEVICE *  pNetIF;

  // Don't add if we have existing route for this target IP address on this interface
	if (StaticRouteExists(IPTarget, IfIdx, diagsOn))
		return NULL;

	pNetIF = NIMUFindByIndex(IfIdx);

	if (diagsOn)
	{
		if (IPGateway)
			LogDiagMessage("IF-%d:%d.%d.%d.%d Static route add GW %d.%d.%d.%d", IfIdx,
					 (UINT8)(IPTarget >> 0) & 0xFF, (UINT8)(IPTarget >> 8) & 0xFF,
					 (UINT8)(IPTarget >> 16) & 0xFF, (UINT8)(IPTarget >> 24) & 0xFF,
					 (UINT8)(IPGateway >> 0) & 0xFF, (UINT8)(IPGateway >> 8) & 0xFF,
					 (UINT8)(IPGateway >> 16) & 0xFF, (UINT8)(IPGateway >> 24) & 0xFF);
		else
			LogDiagMessage("IF-%d:%d.%d.%d.%d Static route add", IfIdx,
								 (UINT8)(IPTarget >> 0) & 0xFF, (UINT8)(IPTarget >> 8) & 0xFF,
								 (UINT8)(IPTarget >> 16) & 0xFF, (UINT8)(IPTarget >> 24) & 0xFF);
	}

	// Start calling STACK functions
  llEnter();

  // Create the route and make it STATIC
  hRt = RtCreate( FLG_RTF_REPORT, FLG_RTE_HOST | FLG_RTE_STATIC, IPTarget, 0xFFFFFFFF, pNetIF, IPGateway, 0 );

  // Since the route is STATIC, we can DeRef it here, and we
  // don't have to worry about keeping track of it.
  if( hRt )
  	RtDeRef( hRt );

  // Stop calling STACK functions
  llExit();

  // Return an indication of success
  return( hRt );
}

To check if route already exists

//=========================================================================
// StaticRouteExists()
//
// Checks if a route already exists
//=========================================================================
int StaticRouteExists( IPN IPTarget , int IfIdx, uint16_t diagsOn)
{
	HANDLE 					hRt;
	HANDLE 					hIF;
	uint   					wFlags;
	UINT32 					IPAddr;
	NETIF_DEVICE *  pNetIF;
	int    					resp = 0;

	pNetIF = NIMUFindByIndex(IfIdx);

	// Start calling STACK functions
	llEnter();

	// Start walking the tree
	hRt = RtWalkBegin();

	// Search while there's more to search
	while (hRt)
	{
		IPAddr = RtGetIPAddr(hRt);
		wFlags = RtGetFlags(hRt);
		hIF    = RtGetIF(hRt);
		// The IP target must match and it must be STATIC HOST
		if((IPTarget == IPAddr) &&
			 ((wFlags & (FLG_RTE_STATIC | FLG_RTE_HOST)) == (FLG_RTE_STATIC | FLG_RTE_HOST)))
		{
			// Rule already exists on this interface
			if (hIF == pNetIF)
			{
				if (wFlags & FLG_RTE_UP)
					resp = 1;
				else
				{
					if (diagsOn)
						LogDiagMessage("IF-%d:%d.%d.%d.%d Remove Route - Not UP", IfIdx,
													 (UINT8)(IPTarget >> 0) & 0xFF, (UINT8)(IPTarget >> 8) & 0xFF,
													 (UINT8)(IPTarget >> 16) & 0xFF, (UINT8)(IPTarget >> 24) & 0xFF);
					// Route is down so we need to remove it
					RtRemove( hRt, FLG_RTF_REPORT, RTC_NETUNREACH );
				}
			}
			else
			{
				if (diagsOn)
					LogDiagMessage("IF-%d:%d.%d.%d.%d Route removed from other IFACE", IfIdx,
												 (UINT8)(IPTarget >> 0) & 0xFF, (UINT8)(IPTarget >> 8) & 0xFF,
												 (UINT8)(IPTarget >> 16) & 0xFF, (UINT8)(IPTarget >> 24) & 0xFF);
				// Same Target is on different interface so remove it
				RtRemove( hRt, FLG_RTF_REPORT, RTC_NETUNREACH );
			}
			break;
		}
		hRt = RtWalkNext( hRt );
	}

	// Finish calling STACK functions
	RtWalkEnd( hRt );

	llExit();
	// Return with found count
	return( resp );
}

Hi Barry,

I apologize for the lack of response, I have been out on leave and just saw your posts. I'm glad you were able to get past this issue.

Cheers,

Steve

Barry,

Thanks, all is going well so far :)

Yes, the NDK is being continuously improved with bug fixes and new features, in parallel with TI-RTOS development.  This includes reviewing the list of open bugs and prioritizing fixes for upcoming releases.  I'll be sure to review this issue during our next planning meeting.

Cheers,

Steve

Barry,

This thread was very helpful.  I also have an application with an ethernet and PPP interface but have an additional requirement I'm struggling with.  I have UDP socket that I would like to receive packets on from either interface.  This I have working.  I then need to be able to send a reply back to the IP address that sent the packet using the same interface that I received the packet on.  Using revncfrom(), I see no way to determine which interface the packet came in on.  Is there some other way to determine this?  If not, maybe adding the socket option IP_PKTINFO. 

Another enhancement might be to have the socket remember the last interface it received on and use that interface when transmitting automatically.  I've worked with another networking stack that does this which is very convenient.    

Also, I'm using NDK_2_25_00_09 and didn't see any socket option SO_BINDTODEVICE yet.  Do you know if this got accepted to implement?  

Thanks

Hello Steve

I don't know if SO_BINDTODEVICE was accepted to implement.

Would be good if it was because would make my code simpler and reduce the need to keep adding and removing routes.

Not sure as to how to go about finding if it will be implemented and if so when.

Have you tried looking at the route table after you receive the packet to see if a rule has been added for that source address?

If one has been added by the receive process this might allow you to identify the interface you received the packet from.

In my case I need to be able to verify the availability of each link independently, hence the need to target the specific interface.

Thanks for the response Steve.

I like the suggestion and will give it a try.

Just one question. How are you determining which route you wish to use and guaranteeing it is via the target interface?

Barry,

In my case I set the hRoute parameter everytime I receive a packet from my host so that my reponse packet goes back to that same host.  Here is how I determine what interface the packet comes in on.  Once I know the interface, I walk the route table to find the correct route on that interface. 

SB   *pSB;

SOCK *ps;  // Socet the packet is received on

int ifId;

pSB = (SB *) ps->hSBRx;

ifId = IFGetIndex(pSB->pPktLast->hIFRx);

/* ifId = 1 is ethernet, ifId = 2 is PPP  */

As far as the refCount, I'm not sure it matters if I do anything with it or not.  In my case, I don't mind if the PPP route just stays

forever whether I use it or not.  I don't think there is anything that is going to decrement the count that would cause it to go away.