PROCESSOR-SDK-AM69: Using the host tools for statistics visualization

Daisuke Maeda

Guru 20255 points

Tool/software:

Our customer wants to try out "Browser-based visualization", but cannot find the host tools for statistics visualization in the latest PSDK Linux 10.01.08.01.

software-dl.ti.com/.../How_to_visualize_statistics_data.html

Does the latest PSDK Linux support the tools? If so, where can we find them?

Best regards,

Daisuke

8 months ago

0 Daisuke Maeda 8 months ago

Guru 20255 points

Hi,

The SDK documentation describes that PSDK Linux installed has the host tools for statistics visualization under the directory "board-support/host-tools/statistics" and uses the "Stats.html" file for "Browser-based visualization", but I can not find that folder and file under the host directory installed.

Does the latest PSDK Linux support the host tools for "Browser-based visualization"? If so, where can we find them?

Best regards,

Daisuke

0 Daisuke Maeda 8 months ago in reply to Daisuke Maeda

Guru 20255 points

Dear TI support team,

Thank you for your daily support. Our customer is waiting for your reply.

Please give me an answer as soon as possible. Your prompt reply would be appreciated.

Best regards,

Daisuke

0 Takuma Fujiwara 8 months ago in reply to Daisuke Maeda

TI__Guru 53533 points

Hi Daisuke-san,

Apologies for the late reply. So far, I can only find the Stats.html website packaged in the 8.6 SDK last, which I will attach below:

Fullscreen Stats.html Download

<!DOCTYPE html>

<head>
	<!--
	<script src="https://cdn.plot.ly/plotly-latest.min.js"></script>
	-->
	<meta content="text/html;charset=utf-8" http-equiv="Content-Type">
	<meta content="utf-8" http-equiv="encoding">
	<script src="https://cdn.plot.ly/plotly-latest.min.js"></script>
	<link href="https://fonts.googleapis.com/css?family=Open+Sans&display=swap" rel="stylesheet">
</head>

<style>
	:invalid {
		border: solid 2px rgb(199, 34, 19);
	}

	.text {
		font-family: 'Open Sans', sans-serif;
		color: rgb(25, 51, 76);
	}

	.graph {
		width: 1000px;
		height: 400px;
		border: 3px solid black;
		border-left: 10px solid rgb(67, 199, 240);
		margin: 10px;
	}

	.graph_caption {
		font-family: 'Open Sans', sans-serif;
		color: rgb(25, 51, 76);
		;
		margin-left: 50px;
		padding-bottom: 50px;
	}

	.indicator {
		width: 600px;
		height: 400px;
		border: 3px solid black;
		border-left: 10px solid rgb(67, 199, 240);
		margin: 10px;
	}

</style>

<body style="margin-left: 20px; margin-right: 20px; background: rgb(220, 220, 220)">
	<div style="display: flex; margin-bottom: -25px;">
		<img src="https://www.ti.com/content/dam/ticom/images/identities/ti-brand/ti-stk-2c-pos-rgb-logo.png"
			style="height: 100px;margin-top: -15px;"></img>
		<h1 style="align-content: center">Statisitcs Visualization</h1>
	</div>
	<hr style="border: rgb(0, 77, 165) solid 2px;"></hr>

	<div style="zoom:1.2;">
		<text style="margin-left:10px;">IP address of the target: </text>
		<input id="ipaddr" autofocus required required pattern="^([0-9]{1,3}\.){3}[0-9]{1,3}$"
			placeholder="xxx.xxx.xxx.xxx" oninvalid="this.setCustomValidity('Invalid IP address!')"
			oninput="this.setCustomValidity('')"></input>
		<text style="margin-left:10px;">Port number: </text>
		<input id="port" value="8090" style="width:70px;"></input>
		<button style="margin-left: 10px;" onclick="connect()">Connect</button>
	</div>
	<div style="margin: 5px;">
		<text id="debug" style="color: brown">Waiting for connection...</text>
	</div>

	<hr style="border: rgb(0, 77, 165) solid 2px;"></hr>
	<div style="display: flex; padding: 3px; margin: 3px;">
		<text>Click to toggle filters: </text>
		<text id="sample_toggle" style="cursor: pointer; padding: 2px; margin-left: 10px; display: none;" target="dummy"
			onclick="toggle_item(this)">Sample</text>
		<div id="toggles" style="display: flex;"></div>
	</div>
	<div id="stats_area" style="border: rgb(0, 77, 165) dashed 2px; min-height: 300px;"></div>
</body>
<script>

	var debug = document.getElementById("debug")
	var linegraphs = {};
	var bargraphs = {};
	var indicators = {};
	var connection;
	var unitTestStop = 0;

	function d(msg) {
		debug.innerHTML = msg
	}

	///////////////////////////////////////////////////////////////////////////
	function isObject(item) {
		return (item && typeof item === 'object' && !Array.isArray(item));
	}

	function mergeDeep(target, ...sources) {
		if (!sources.length) return target;
		const source = sources.shift();

		if (isObject(target) && isObject(source)) {
			for (const key in source) {
				if (isObject(source[key])) {
					if (!target[key]) Object.assign(target, { [key]: {} });
					mergeDeep(target[key], source[key]);
				} else {
					Object.assign(target, { [key]: source[key] });
				}
			}
		}
		return mergeDeep(target, ...sources);
	}

	function convert_units_to_absolute(input) {
		var regex = /([0-9]+)([KMG])bps/;
		var conversion = { "K": 1000, "M": 1000000, "G": 1000000000 };
		output = []
		//console.log("input = " + input);
		for (value of input) {
			match = regex.exec(value);
			if (match) {
				value = match[1] * conversion[match[2]];
			}
			output.push(value);
		}
		//console.log("output = " + output);
		return output;
	}

	///////////////////////////////////////////////////////////////////////////
	function linegraphCreate(id, caption, traces, num_samples, custom_layout) {
		var graph = new Object();
		graph.id = id;
		graph.num_samples = num_samples;
		graph.cnt = 0

		parent = document.getElementById('stats_area');
		div = document.createElement("div");
		div.setAttribute('id', id);
		div.className = "graph";
		parent.appendChild(div);
		graph.div = div;

		data = traces;

		layout = {
			plot_bgcolor: "#F8F8F8",
			showlegend: true,
			title: {
				text: caption,
				font: {
					family: 'Open Sans, sans-serif',
					color: '#7f7f7f',
				},
				xref: 'paper',
				x: 0.05,
			},
			xaxis: {
				range: [0, graph.num_samples],
				title: {
					text: 'Time',
					font: {
						family: 'Open Sans, sans-serif',
						color: '#7f7f7f',
					},
				},
			},
			yaxis: {
				title: {
					text: caption,
					font: {
						family: 'Open Sans, sans-serif',
						color: '#7f7f7f',
					},
				},
			},
		};

		if (custom_layout) {
			mergeDeep(layout, custom_layout);
		}
		Plotly.plot(graph.id, data, layout);
		return graph;
	}

	function linegraphUpdate(graph, data) {
		var indices = [];
		for (i = 0; i < data.length; i++) {
			indices.push(i);
		}
		Plotly.extendTraces(graph.id, { y: data }, indices);
		graph.cnt++;

		if (graph.cnt > graph.num_samples) {
			Plotly.relayout(graph.id, {
				xaxis: {
					range: [graph.cnt - graph.num_samples, graph.cnt]
				}
			});
		}
	}

	function handle_linegraph(values, id, name, label, custom_layout) {
		var traces = [];
		var data = [];
		var x;

		if (linegraphs[id] == null) {
			for (i = 0; i < parseInt(values[1]); i++) {
				type = values[2 + i * 2];
				val = parseInt(values[3 + i * 2]);
				console.log("Adding linegraph trace for " + id + ":" + type);
				traces.push({
					y: [0],
					name: type,
					type: 'line',
				});
				data.push([val,]);
			}
			linegraphs[id] = linegraphCreate(id, label, traces, 100, custom_layout);
			linegraphUpdate(linegraphs[id], data);
			add_toggle_button(id, name);
		} else {
			for (i = 0; i < parseInt(values[1]); i++) {
				val = values[3 + i * 2];
				data.push([val,]);
			}
			linegraphUpdate(linegraphs[id], data);
		}
	}

	///////////////////////////////////////////////////////////////////////////
	function bargraphCreate(id, caption, x, y, custom_layout) {
		var graph = new Object();
		width = [];
		graph.id = id;

		parent = document.getElementById('stats_area');
		div = document.createElement("div");
		div.setAttribute('id', id);
		div.className = "graph";
		parent.appendChild(div);
		graph.div = div;

		for (val of y) {
			width.push(0.3);
		}
		console.log(width);
		data = [{
			x: x,
			y: y,
			type: 'bar',
			name: caption,
			width: width,
		}];
		layout = {
			yaxis: {
				range: [0, 100],
				title: {
					text: caption,
					font: {
						family: 'Open Sans, sans-serif',
						color: '#7f7f7f',
					},
				},
			},
		};

		if (custom_layout) {
			mergeDeep(layout, custom_layout);
		}
		Plotly.plot(graph.id, data, layout);
		return graph;
	}

	function bargraphUpdate(graph, y) {
		Plotly.restyle(graph.id, 'y', [y]);
	}

	function handle_bargraph(values, id, name, custom_layout) {
		var x = [];
		var y = [];

		if (bargraphs[id] == null) {
			for (i = 0; i < parseInt(values[1]); i++) {
				type = values[2 + i * 2];
				val = parseInt(values[3 + i * 2]);
				console.log("Adding bargraph for " + id);
				x.push(type);
				y.push(val);
			}
			bargraphs[id] = bargraphCreate(id, name, x, y, custom_layout);
			add_toggle_button(id, name);
		} else {
			for (i = 0; i < parseInt(values[1]); i++) {
				val = parseInt(values[3 + i * 2]);
				y.push(val);
			}
			bargraphUpdate(bargraphs[id], y);
		}
	}
	///////////////////////////////////////////////////////////////////////////
	function indicatorCreate(id, name, maxval) {
		var indicator = new Object();
		indicator.id = id;
		indicator.maxval = maxval;

		parent = document.getElementById('stats_area');
		div = document.createElement("div");
		div.setAttribute('id', id);
		div.className = "indicator";
		parent.appendChild(div);
		indicator.div = div;
		indicator.data = [
			{
				domain: { x: [0, 1], y: [0, 1] },
				value: 45,
				title: { text: name },
				type: "indicator",
				mode: "gauge+number",
				gauge: { axis: { range: [0, maxval] } }
			}
		];
		indicator.layout = { width: 600, height: 400 };
		Plotly.newPlot(indicator.id, indicator.data, indicator.layout);
		return indicator;
	}

	function indicatorUpdate(indicator, value) {
		indicator.data[0].value = value;
		Plotly.newPlot(indicator.id, indicator.data, indicator.layout);
	}

	function handle_indicator(values, id, name, label, maxval) {
		if (indicators[id] == null) {
			console.log("Adding indicator for " + name);
			indicators[id] = indicatorCreate(id, label, maxval);
			add_toggle_button(id, name);
		} else {
			indicatorUpdate(indicators[id], parseInt(values[1]));
		}
	}

	///////////////////////////////////////////////////////////////////////////
	function reset() {
		if (connection != null) {
			connection.close();
		}

		linegraphs = {};
		indicators = {};

		document.getElementById('stats_area').innerHTML = "";
		document.getElementById('toggles').innerHTML = "";
		d("Waiting for connection...");
	}

	function handle_message(event) {
		//console.log("Received data: " + event.data);
		values = event.data.split(" ");
		tag = String(values[0]);

		switch (tag) {
			case "WS-cpuload:":
				layout = {
					yaxis: {
						range: [0, 100],
					},
				};
				handle_linegraph(values, 'cpuload', "CPU load", "CPUload (%)", layout);
				handle_bargraph(values, 'bar_cpuload', "CPU load bar", "CPUload (%)", layout);
				break;
			case "WS-ddrbw:":
				layout = {
					yaxis: {
						range: [0, 10000],
					},
				};
				handle_linegraph(values, 'ddrbw', "DDR Bandwidth", "DDR bandwidth (MBps)", layout);
				break;
			case "WS-ethfw-bw:":
				values = convert_units_to_absolute(values);
				layout = {
					yaxis: {
						range: [0, 9],
						type: "log",
					},
				};
				handle_linegraph(values, 'ethfwbw', "Ethernet bandwidth", "Ethernet bandwidth", layout);
				break;
			case "WS-cluster-fps:":
				handle_indicator(values, 'cluster-fps', "Cluster FPS", "Cluster FPS", 60)
				break;
			default:
				console.log("Unsupported tag " + tag);
		}
	}

	function connect() {

		unitTestStop = 1;
		reset();

		ipaddr = document.getElementById('ipaddr').value
		port = document.getElementById('port').value
		connection = new WebSocket('ws://' + ipaddr + ':' + port);

		connection.onopen = function () {
			d("Connected to " + ipaddr);
		};

		connection.onmessage = handle_message;
	}

	function add_toggle_button(id, text) {
		parent = document.getElementById('toggles');
		node = document.getElementById('sample_toggle');
		newtoggle = node.cloneNode(true);

		delete newtoggle.id;
		newtoggle.innerHTML = text;
		newtoggle.setAttribute('target', id);
		newtoggle.style.display = "block";
		newtoggle.style.backgroundColor = "#3273dc4d";
		parent.appendChild(newtoggle);
	}

	function toggle_item(element) {
		id = element.getAttribute('target');
		console.log("Clicked on " + id);
		item = document.getElementById(id);
		if (item == null) {
			return
		}
		if (item.style.display == "none") {
			item.style.display = "block";
			element.style.backgroundColor = "#3273dc4d";
		} else {
			item.style.display = "none";
			element.style.backgroundColor = "#f2f2f2";
		}
	}

	///////////////////////////////////////////////////////////////////////////
	function sleep(ms) {
		return new Promise(resolve => setTimeout(resolve, ms));
	}

	function testMsg(msg) {
		handle_message({ data: msg });
	}
	function unitTest() {

		function rand(min, max) {
			diff = max - min
			return Math.floor((Math.random() * diff) + min);
		}

		function test_random_data() {

			d("Testing with random data, NOT connected to any target!");
			if (unitTestStop == 1)
				return
			testMsg("WS-cpuload: 4 R5F " + rand(33, 62) + " C6x " + rand(52, 59) + " A72 " + rand(9, 42) + " GPU " + rand(73, 91));
			testMsg("WS-ddrbw: 4 Read_avg " + rand(1800, 2200) + " Write_avg " + rand(900, 1300) + " Read_peak " + rand(4000, 4400) + " Write_peak " + rand(3100, 4200));
			testMsg("WS-ethfw-bw: 4 Port0_RX " + rand(0, 999) + "Kbps Port0_TX " + rand(150, 300) + "Kbps Port1_RX " + rand(300, 700) + "Mbps Port1_TX " + rand(12, 67) + "Mbps");
			testMsg("WS-cluster-fps: " + rand(45, 60));
			setTimeout(test_random_data, 300);
		}

		document.getElementById('ipaddr').value = "172.24.190.9";
		test_random_data();
	}

	///////////////////////////////////////////////////////////////////////////
	reset();
	
	//unitTest();

</script>

</html>

Currently checking with the team whether this was descoped from our SDK. I will need 1~2 business days to confirm with the broader global team.

Regards,

Takuma

0 Takuma Fujiwara 8 months ago in reply to Takuma Fujiwara

TI__Guru 53533 points

Hi Daisuke-san,

I have gathered information from the team for the statcol tool. It looks like this tool is no longer being maintained in recent releases.

So unfortunately, no guarantees that this tool is still functional.

As for some alternatives:

If using TI's GStreamer components from EdgeAI SDK, then there are some plugins that can be used for displaying performance data: https://software-dl.ti.com/jacinto7/esd/processor-sdk-linux-am69a/10_01_00/exports/edgeai-docs/common/measure_perf.html.

Regards,

Takuma

0 Daisuke Maeda 8 months ago in reply to Takuma Fujiwara

Guru 20255 points

Hi Takuma-san,

Thank you for your reply.

Takuma Fujiwara said:
If using TI's GStreamer components from EdgeAI SDK, then there are some plugins that can be used for displaying performance data: https://software-dl.ti.com/jacinto7/esd/processor-sdk-linux-am69a/10_01_00/exports/edgeai-docs/common/measure_perf.html.

Do the EdgeAI SDK and the GStreamer components support AM69x without AI accelerator?

Our customer is using PSDK Linux for AM69x but not AM69Ax.

Best regards,

Daisuke

0 Takuma Fujiwara 8 months ago in reply to Daisuke Maeda

TI__Guru 53533 points

Hi Daisuke-san,

There is dependency on the underlying RTOS running on the remote cores to run TI OpenVX, since the ti_perfstats and visualizer utilize the API here: https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-j784s4/10_01_00_04/exports/docs/vision_apps/docs/user_guide/group__group__vision__apps__utils__perf__stats.html

Although, you mention customer is using PSDK Linux for AM69x, but does this mean customer is only running Linux on A72 main CPU core, with no firmware flashed on R5F, C7x, and other remote cores? If this is the case, then we would only care about the A72 cores, and Linux has tools like "top" to measure CPU/memory usage.

Regards,

Takuma

0 Daisuke Maeda 8 months ago in reply to Takuma Fujiwara

Guru 20255 points

Hi Takuma-san,

Thank you for your reply.

I understand that to achieve statistics collection in the same way as the Vision Apps in PSDK Linux for TDA4 (J784S4), PSDK RTOS (FreeRTOS) for TDA4 (J784S4) or an equivalent OS needs to be running on the remote cores (R5F and C7x).

software-dl.ti.com/.../index.html

Our customer does not need the graphical visualizations displayed in the Vision Apps demo, the text information displayed in ti_perfstats will suffice. It would be even better if it could be visualized graphically on the host side, something like "browser-based visualization".

git.ti.com/.../README.md

Our customer may use a third-party RTOS for R5F, but will use default firmware to test PSDK Linux.

Is there any other way to achieve statistics collection and display when using the PSDK Linux for AM69x with the default firmware?

Best regards,

Daisuke

0 Takuma Fujiwara 8 months ago in reply to Daisuke Maeda

TI__Guru 53533 points

Hi Daisuke-san,

All of the examples that we have, including the browser-based visualization, GStreamer plugin, and perfstat, all depend on vision apps to be running on remote cores to collect statistics from R5F and C7x.

If a third-party RTOS is running, then the third party must implement some way to collect statistics on the remote cores, similar to the TI Vision Apps examples. For example, in the vision apps examples, CPU load of remote cores are obtained from calculations using the time a CPU is in the idle task, and the total time available.

Regards,

Takuma

Processors

Processors forum

PROCESSOR-SDK-AM69: Using the host tools for statistics visualization