Codec Engine, Framework Components on Windows

Hey, Matt.  I moved this to the BIOS forum for lack of a better place... I think it's where the interested parties hang out, though.

We may also have a friend here:  http://e2e.ti.com/support/embedded/bios/f/355/t/163271.aspx

I'm certainly interested in where this goes.  I manage CE, FC, OSAL and have some input to the XDC folks, so I can probably help a little from the "TI inside".

My team uses the gnu.targets.Linux86 target to prototype/test our portable products on Linux-based PCs, and I've also seen interest in Windows hosted libraries, too, so I'm definitely interested.  Related, XDC historically has limited, best-effort .dll support, so we may learn some things together and help advance the platform in that respect, too.

It's certainly worth collaborating on this - especially if you're going to do the work anyway.  I have limited resources to invest in this, but can at least help guide and strategize.  Any problem with licensing your/our collaborative work with an open license like BSD?  Any interest in putting together a little collaborative project on github?

Chris

Matt,
a couple of years ago I spent some time working on the microsoft.targets package, and we are actually using these targets to build some DLLs shipped in the XDCtools product. However, the work on these targets mainly stopped for the lack of customers interested in them. However, I am certainly interested in reviving them, or replacing them with something better.

Now, I am not really qualified to talk about licensing details, but I am sure we can find someone at TI who can help. From the technical point of view, we certainly welcome any new developed RTSC targets, but if we deliver them with XDCtools we need to be comfortable with supporting them.

It seems you managed to get shared libraries built on Linux. Is that correct? Can you give us more details? I just want to compare it with what I did. Here is some code that I use to build DLLs for microsoft.targets.VC98, but it could be easily turned into support for microsoft.targets.Win32. The code contains some undocumented features, so I am posting it here not as an example of how it should be done, but rather as a reminder of the issues that are not solved in XDCtools, so I had to work around them.

The example builds a DLL contains Mod1.c, Mod2,c and Mod3.c sources. First, here is package.bld, where you declare what you want to build:

var SOURCES = ["Mod1.c", "Mod2.c", "Mod3.c"]; // .etc
for (var i = 0; i < Build.targets.length; i++) {
    var targ = Build.targets[i];
    if (targ.name != "VC98") {return;} // it could probably work for some other Windows targets, but I tested it only with VC98

    /* First, a static library 'stat' is built, whose purpose is to build object files, which are used as an input to build a DLL 'dyn'. */
    var libr = Pkg.addLibrary("release/stat", targ, {profile: "release", suffix: ".lib"});
    libr.addObjects(SOURCES);

    var dllr = Pkg.addAssembly("release/dyn", targ, targ.platform, {profile: "release"});

    /* Add 'res' and 'def' files to the command line */
    dllr.attrs.lopts += " release/dyn.res -def:dyn.def ";

    /* We need to add a 'rc' command to makefile because RTSC targets do not support resource compilation as a step in the build process. */
    var inc = xdc.module("microsoft.targets.VC98").Module(targ).includeOpts;
    inc = inc.replace(/rootDir/g, "microsoft.targets.VC98.rootDir");

    makeAdd = ".libraries: release/dyn.res\n" +  "release/dyn.res: dyn.rc\n\t" +
                         xdc.module("microsoft.targets.VC98").Module(targ).cmdPrefix +
                        "$(microsoft.targets.VC98.rootDir)/common/msdev98/bin/rc " +
                        inc + " $(XDCINCS) /fo $@ $<\n\n";
}

Pkg.makeEpilogue = makeAdd;
Pkg.makeEpilogue += "clean::\n\t-$(RM) -f " + "release/dyn.res\n";
Pkg.attrs.exportExe = true;

Then, in package.xs we need to return the object files when getLibs() is called, but we do that only the assembly (DLL) is being built. We recognize that case by checking Program.$$isasm (undocumented feature).

function getLibs()
{
    if (Program.build.target.name != "VC98") {return;}

    /* *.o files are returned only when bioscfg.dll is built within this package. */
    if (Program.$$isasm == 1 && Program.buildPackage == this.$name) {
        return ("package/lib/release/stat/Mod1.o" + Program.build.target.suffix +
             ";package/lib/release/stat/Mod2.o" + Program.build.target.suffix +
             ";package/lib/release/stat/Mod3.o" + Program.build.target.suffix);
    }
    else {
        return null;
    }
}

Hey Chris,

That is a good link.  Pity it was tied under a clients licensing.  We're pretty happy here to do a really open license - we don't like any type of restrictive license, so BSD is cool with us (or WTFPL! ;) )

I'll give you a warning that this is scheduled in over the next 6 months so can't give you any time scale of when this work will actually start!  It's great to have some support though :) and we need to do it so solve a lot of our problems so will definitely happen.

We use the gnu.targets.Linux86 target to prototype/test too.  It's really useful.  To be able to plop out stuff on windows would be such a massive benefit!  I guess a lot of the work that was done on the components to get them running on a host Linux machine will transfer across.  I figure that there will be quite a bit of work on the OSAL - strangely enough we are looking at writing our own OSAL for a new project we are doing so could maybe invest the effort there in improving what you've got.

I wouldn't worry to much about your limited resources, we are happy to do the work, this would be a much better solution than us writing my own DVSDK abstraction layer.

I'm happy to contribute to XDC, I've spoke to Dave Russo on the forums before about splint and doxygen XDC integration, which went really stale because we had to shift focus, but we're coming out of that now and looking to get our build working on all platforms robustly with SCA and documentation.

I'm happy with doing something on github.  Is the XDC component source code SVN or git bound?

So the first question is - how would I get the XDC components source code and build it?!  The DVSDK components say they are missing the package.bld.

Matt

Sasha,

I do not have shared libraries building - they all are static libraries *.a86U.  I thought this was the way that XDC was supposed to work?  When you pull everything together in the engine configuration, it statically links all that stuff into one executable.  I maybe way off there.

I've added my config.bld, common_package.bld and package.bld below.  These are used to set up the build.  I found a lot of the TI examples required a lot of manual changes to the XDC build so tried to make it as automatic as possible.  The folder structure in our build looks like:

.
|-- doc
|-- inc
|   `-- header.h
|-- priv
|   `-- private_header.h
'-- src
    |-- platform
    |   |-- generic
    |   |   '-- generic_c_kernel.c
    |   |-- etc
    |   '-- C64P
    |       '-- C64P_optimised_kernel.c
    `-- source_file.c

And the platform folder allows use to optimised bits of code for the various platforms.  These are my best effort - if I'm doing anything stupid please tell me!

Is that what you wanted Sasha or have I just created a really long useless post!!

Matt

Return to index

/*
 * Please use a tab size of 8.
 */

/*
 * This file uses the Qt style of documentation. Read about it here:
 * http://www.stack.nl/~dimitri/doxygen/manual.html
 */

/*! \file	config.bld
 *  \brief	The XDC build configuration script.
 *  \details	This script is run prior to all build scripts.  It initializes the
 *		rootDir configuration parameter of all supported targets.
 *
 *  \see	http://processors.wiki.ti.com/index.php/Config_bld_basics
 */

/*jslint
	continue	: true,
*/
/*global
	xdc,
	print
*/

var Build	= xdc.useModule("xdc.bld.BuildEnvironment");

// initialize local vars with those set in xdcpaths.mak (via XDCARGS)
var x = 0;
var cgRootDir		= false;
var mvRootDir_V5T_le	= false;
var linux86RootDir	= false;
var platforms		= false;
var intermediateFiles	= false;
var verbose		= false;
var debug		= false;
var biosAvailable	= false;
for(x = 0; x < arguments.length; x++)
{
	if (arguments[x].match(/^CODEGEN_INSTALL_DIR=/))
	{
		cgRootDir = arguments[x].split('=')[1];
	}
	else if (arguments[x].match(/^MONTAVISTA_V5T_LE_TOOLS_DIR=/))
	{
		mvRootDir_V5T_le = arguments[x].split('=')[1];
	}
	else if (arguments[x].match(/^LINUX86_TOOLS_DIR=/))
	{
		linux86RootDir = arguments[x].split('=')[1];
	}
	else if (arguments[x].match(/^PLATFORMS=/))
	{
		platforms = arguments[x].split('=')[1].split(' ');
	}
	else if (arguments[x].match(/^INTERMEDIATE_FILES=/))
	{
		intermediateFiles = (arguments[x].split('=')[1].toLowerCase() === "yes");
	}
	else if (arguments[x].match(/^VERBOSE=/))
	{
		verbose = (arguments[x].split('=')[1].toLowerCase() === "yes");
	}
	else if (arguments[x].match(/^DEBUG=/))
	{
		debug = (arguments[x].split('=')[1].toLowerCase() === "yes");
	}
	else if (arguments[x].match(/^BIOS_AVAILABLE=/))
	{
		biosAvailable = (arguments[x].split('=')[1].toLowerCase() === "yes");
	}
}

// Construct the build targets array
var i = 0;
for(i = 0; i < platforms.length; i++)
{
	// The target to build
	var target;
	
	// A flag to make sure we don't add duplicate targets
	var addTarget	= true;

	// Decide which targets(s) we are building
	switch(platforms[i])
	{
		case "C64P":
			if(!biosAvailable)
			{
				print("There is no BIOS available in this DVSDK, the " + platforms[i] + " will not be built");
				continue;
			}
			target			 = xdc.useModule("ti.targets.C64P");
			target.rootDir		 = cgRootDir;
			target.ccOpts.prefix	+= " -mi10";
			target.ccOpts.prefix	+= " -mo";
			if(intermediateFiles)
			{
				target.ccOpts.prefix	+= " -k";
				target.ccOpts.prefix	+= " -s";
				target.ccOpts.prefix	+= " -os";
				target.ccOpts.prefix	+= " -on=2";
				target.ccOpts.prefix	+= " --consultant";
				target.ccOpts.prefix	+= " --no_compress";
				target.ccOpts.prefix	+= " -mw";
			}
			break;

		case "MVArm9":
			target			 = xdc.useModule("gnu.targets.MVArm9");
			target.rootDir		 = mvRootDir_V5T_le;

			target.ccOpts.prefix	+= " -Wall";
			target.ccOpts.prefix	+= " -Wextra";
			target.ccOpts.prefix	+= " -std=c99";
			target.ccOpts.prefix	+= " -pedantic";
			target.ccOpts.prefix	+= " -Wno-missing-field-initializers";	// For struct = {0}
			target.ccOpts.prefix	+= " -Wno-missing-braces";		// For struct = {0}
			if(!debug)
			{
  				target.ccOpts.prefix	+= " -pedantic-errors";
 				target.ccOpts.prefix	+= " -Werror";
			}
			if(intermediateFiles)
			{
				target.ccOpts.suffix	+= " -save-temps";
			}
			break;

		case "Linux86":
			target			 = xdc.useModule("gnu.targets.Linux86");
			target.rootDir		 = linux86RootDir;

			target.ccOpts.prefix	+= " -Wall";
			target.ccOpts.prefix	+= " -Wextra";
			target.ccOpts.prefix	+= " -std=c99";
			target.ccOpts.prefix	+= " -pedantic";
			target.ccOpts.prefix	+= " -Wno-missing-field-initializers";	// For struct = {0}
			target.ccOpts.prefix	+= " -Wno-missing-braces";		// For struct = {0}
			/* This is a fail on XDC part. It will build for 64bit on 64bit architectures
			 * we need to force 32bit archives to be compatible
			 */
			target.ccOpts.prefix	+= " -m32";				// Must build for 32 bit
			target.lnkOpts.prefix	+= " -m32  -lpthread ";			// Must build for 32 bit
			if(!debug)
			{
				target.ccOpts.prefix	+= " -pedantic-errors";
				target.ccOpts.prefix	+= " -Werror";
			}
			if(intermediateFiles)
			{
				target.ccOpts.prefix	+= " -save-temps";
			}
			break;

		default:
			print('Warning: Encountered an unsupported platform (' + platforms[i] + ')');
			continue;
	}

	// Add the target define
	target.ccOpts.suffix	+= " -DTARGET_" + platforms[i].toUpperCase();

	// Make the release optimisation -O3 instead of -O2
	if(target.profiles.release)		target.profiles.release.compileOpts.copts	= target.profiles.release.compileOpts.copts.replace('-O2','-O3');
	if(target.profiles.whole_program)	target.profiles.whole_program.compileOpts.copts	= target.profiles.whole_program.compileOpts.copts.replace('-O2','-O3');

	/* Add tracing to the profiles.  We can decide in the future if we want
	 * to release two different profiles, one with tracing and one without
	 * for each profile
	 */
	var j = 0;
	for(j = 0; j < target.profiles.length; j++)
	{
		target.profiles[j].compileOpts.copts	+=	" -DUTILS_TRACE";
//		target.profiles[j].compileOpts.copts	+=	" -DUTILS_TRACE_RUNNING";
	}

	// We remove a few profiles, just to cut down on build time
	delete target.profiles.coverage;
	delete target.profiles.whole_program;
	delete target.profiles.whole_program_debug;

	// Remove the debug if it is not needed
	if(!debug)
	{
		delete target.profiles.debug;
		delete target.profiles.profile;
	}

	// Check we haven't already added this platform
	for(j = 0; j < Build.targets.length; j++)
	{
		if(Build.targets[j].name === target.name)
		{
			print('Found duplicate \'' + target.name + '\'.');
			addTarget = false;
			break;
		}
	}

	// Add the target to the targets if not already added
	if(addTarget)
	{
		Build.targets.$add(target);
	}
}

Return to index

/*
 * Please use a tab size of 8.
 */

/*
 * This file uses the Qt style of documentation. Read about it here:
 * http://www.stack.nl/~dimitri/doxygen/manual.html
 */

/*! \file	build/package.bld
 *  \brief	This file specifies the package build setting for the XDC tools
 */

/*jslint
*/
/*global
	xdc,
	java,
	target,
	sources,
	print
*/
var Build	= xdc.useModule('xdc.bld.BuildEnvironment');
var Pkg		= xdc.useModule('xdc.bld.PackageContents');

// Initialize local vars with those set in xdcpaths.mak (via XDCARGS)
var releaseSource = false;
var x = 0;
for(x = 0; x < arguments.length; x++)
{
	if (arguments[x].match(/^RELEASE_SOURCE=/))
	{
		releaseSource = (arguments[x].split('=')[1].toLowerCase() === "yes");
	}
}

// Adds all files of a certain type in a folder to an array
function pushFiles(array, folder, extension)
{
	var folderList		= folder.listFiles();
	var i = 0;
	for(i = 0; folderList && i < folderList.length; i++)
	{
		if(folderList[i].isFile() && folderList[i].getName().endsWith(extension))
		{
			array.push(folder.toString() + '/' + folderList[i].getName());
		}
	}
}

// Adds all files of a certain type in a folder to an array
function addFiles(array, folder, extension)
{
	var folderList		= folder.listFiles();
	var i = 0;
	for(i = 0; folderList && i < folderList.length; i++)
	{
		if(folderList[i].isFile() && folderList[i].getName().endsWith(extension))
		{
			array.$add(folder.toString() + '/' + folderList[i].getName());
		}
	}
}

// Export internal source code and information. (Only do this if a customer pays a LOT of money :)
Pkg.attrs.exportSrc	= releaseSource;
if(Pkg.attrs.exportSrc)
{
	Pkg.otherFiles.$add('package.bld');	// XDC build script
	Pkg.otherFiles.$add('doc/internal');	// Internal Documentation
	var privFolder = new java.io.File('priv');
	if(privFolder.exists())
	{
		addFiles(Pkg.otherFiles, privFolder, '*.h');
	}
}

// Always include the external documentation
Pkg.otherFiles.$add('doc/external');

// Add the header files in the inc folder
var publicHeaders	= [];
var incFolder		= new java.io.File('inc');
pushFiles(publicHeaders, incFolder, '.h');
addFiles(Pkg.otherFiles, incFolder, '.h');

// Add the source files in the src folder to be built
sources	= [];
var srcFolder	= new java.io.File('src');
pushFiles(sources, srcFolder, '.c');
pushFiles(sources, srcFolder, '.cpp');

// Add the sources in the platform specific folder
var platformSrcFolder	= new java.io.File('src/platform');
if(platformSrcFolder.exists())
{
	print('Adding platform specific code for ' + target.name);
	platformSrcFolder	= new java.io.File('src/platform/' + target.name);
	if(srcFolder.exists() && !platformSrcFolder.exists())
	{
		print('Warning: Did not find \'src/platform/' + target.name + '\' platform specific code.  Defaulting to the \'src/platform/generic\' code.');
		platformSrcFolder = new java.io.File('src/platform/generic');
	}
	pushFiles(sources, platformSrcFolder, '.c');
}

// A build function to build a library
function buildLib()
{
	// lib/ is a generated directory that 'xdc clean' should remove
	Pkg.generatedFiles.$add('lib/');

	// Get the library name from the folder name
	var vendorFolder	= new java.io.File('..');
	var moduleFolder	= new java.io.File('.');
	var vendorName		= vendorFolder.getCanonicalPath().match(/[a-z_]+$/).toString();
	var moduleName		= moduleFolder.getCanonicalPath().match(/[a-z_]+$/).toString();

	var i = 0;
	for(i = 0; i < Build.targets.length; i++)
	{
		var target = Build.targets[i];

		print('Building ' + moduleName + ' for ' + target.name);

		// Build for all profiles
		var profile;
		for(profile in target.profiles)
		{
			var libName = 'lib/' + profile + '/' + moduleName;
			var fullLibName = libName + '.a' + target.suffix;

			Pkg.addLibrary(libName, target, {profile: profile}).addObjects(sources);

			if(rootDir !== undefined)
			{
				// Use the QualiTI template to generate .qti scripts
				var qtiAttrs =
					[
						vendorName.toUpperCase(),		// vendor
						publicHeaders,				// headers (relative to base dir)
						'I' + moduleName.toUpperCase(),		// interface name
						moduleName.toUpperCase(),		// module name,
						fullLibName,				// lib name (relative to base dir)
						'../..'					// base dir (relative to .qti script)
					];
	
				xdc.loadTemplate(rootDir + '/build/QtiProject.xdt').genFile(fullLibName + '.qti', Pkg, qtiAttrs);
	
				// Include the newly generated .qti script to the release
				Pkg.otherFiles.$add(fullLibName + '.qti');
			}
		}
	}
}

// A build function to build an executable
function buildExe( )
{
	// lib/ is a generated directory that 'xdc clean' should remove
	Pkg.generatedFiles.$add('exe/');

	// Get the executable name from the folder name
	var vendorFolder	= new java.io.File('..');
	var moduleFolder	= new java.io.File('.');
	var vendorName		= vendorFolder.getCanonicalPath().match(/[a-z_]+$/).toString();
	var moduleName		= moduleFolder.getCanonicalPath().match(/[a-z_]+$/).toString();

	var i = 0;
	for(i = 0; i < Build.targets.length; i++)
	{
		var target = Build.targets[i];

		print('Building ' + moduleName + ' for ' + target.platform);

		// Check if there is a local config script for this target
		var cfgScript = moduleFolder.getCanonicalPath().toString() + '/' + target.name;
		var cfgFile = new java.io.File( cfgScript + '.cfg' );
		
		if( !cfgFile.exists() )
		{
			// Use the default
			cfgScript = rootDir + '/build/xdc_configs/' + target.name;
		}

		// Build for all profiles

		var profile;
		for(profile in target.profiles)
		{
			var exeName = 'exe/' + profile + '/' + moduleName;
			var fullExeName = exeName + '.x' + target.suffix;

			Pkg.addExecutable(exeName, target, target.platform,
			{
				profile:	profile,
				cfgScript:	cfgScript
			}).addObjects(sources);
		}
	}

Return to index

/*
 * Please use a tab size of 8.
 */

/*
 * This file uses the Qt style of documentation. Read about it here:
 * http://www.stack.nl/~dimitri/doxygen/manual.html
 */

/*! \file	udpanalytics/vca_core/package.bld
 *  \brief	This file specifies the package build setting for VCA Core
 */

// Set the root directory
var rootDir	= '../..';

// Load the common package.bld
var common	= xdc.loadCapsule(rootDir + '/build/common_package.bld');

/* You can add extra package specific stuff here to the variables
 * that are set up in the above common file
 */

// Build the package
common.buildLib();

Return to index

Matt Clarkson said:

Is that what you wanted Sasha or have I just created a really long useless post!!

Matt

I don't think it's useless. I wasn't looking for anything specific, I just wanted to see what you've done. I'll take a look at the code, and if you don't mind, I might have some further questions.