Bulijiojio Blog

日常摸鱼划水的虾皮--如果公式不能正常显示请使用Chrome浏览器并安装Tex All the Things插件

0%

前言

在之前的5篇博客中我们介绍了, TI out of box demo, Ti 3D people counting demo 以及使用他们的数据进行的目标跟踪与航迹管理, 还有TI out of box demo的源码程序, 那按照理论上来说, 我们其实就可以在CCS中对mmWave SoC 进行编程从而实现我们自己所设计的信号处理流程, 通过直接去写C语言来对 ARM 和 DSP 编程的方式需要比较长的时间, 而在进行算法验证的初期我们更多是使用 Matlab, Python 等一些高级语言进行算法验证, 在性能达到我们预期的情况下, 我们才会考虑将这样的算法移植到SOC的平台中, 这样能够为我们的开发节省出很多的时间.

Ti 官方已经提供了 mmWaveStudio 这样的工具, 在搭配 iwr设备, mmWave ICBoost承载板卡, DCA1000数据采集办卡后, 我们可以通过LVDS获得雷达各个接收通道的中频信号(也可以叫差拍信号). 使用 mmWaveStudio 工具进行原始数据获取TI官方给了非常详尽的文档, 如果还没有使用过 mmWaveStudio 的读者, 请先使用官方方式进行数据采集然后再来参考本篇博客

/ti/mmwave_studio_02_01_01_00/docs/mmwave_studio_user_guide.pdf

在这篇博客当中, 我将介绍如何使用 python 脚本对 mmWave radar 的原始数据进行获取.

DCA1000EVM Overview

硬件连接

相关代码(python)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
# Copyright 2019 The OpenRadar Authors. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================

import codecs
import socket
import struct
import serial
import time
import matplotlib.pyplot as plt
# import pyqtgraph as pg
# from mpl_toolkits.mplot3d import Axes3D
from enum import Enum
# from matplotlib.backends.backend_qt5agg import FigureCanvasQTAgg as FigureCanvas
# from matplotlib.figure import Figure
# from matplotlib.lines import Line2D
# import matplotlib.cbook as cbook
import numpy as np

configFileName = "IWR6843_cfg.cfg"


class CMD(Enum):
RESET_FPGA_CMD_CODE = '0100'
RESET_AR_DEV_CMD_CODE = '0200'
CONFIG_FPGA_GEN_CMD_CODE = '0300'
CONFIG_EEPROM_CMD_CODE = '0400'
RECORD_START_CMD_CODE = '0500'
RECORD_STOP_CMD_CODE = '0600'
PLAYBACK_START_CMD_CODE = '0700'
PLAYBACK_STOP_CMD_CODE = '0800'
SYSTEM_CONNECT_CMD_CODE = '0900'
SYSTEM_ERROR_CMD_CODE = '0a00'
CONFIG_PACKET_DATA_CMD_CODE = '0b00'
CONFIG_DATA_MODE_AR_DEV_CMD_CODE = '0c00'
INIT_FPGA_PLAYBACK_CMD_CODE = '0d00'
READ_FPGA_VERSION_CMD_CODE = '0e00'

def __str__(self):
return str(self.value)


# MESSAGE = codecs.decode(b'5aa509000000aaee', 'hex')
CONFIG_HEADER = '5aa5'
CONFIG_STATUS = '0000'
CONFIG_FOOTER = 'aaee'
ADC_PARAMS = {
'chirps': 96,
'rx': 4,
'tx': 3,
'samples': 96,
'IQ': 2,
'bytes': 2
}
# STATIC
MAX_PACKET_SIZE = 2097152
# 默认数据包长度
BYTES_IN_PACKET = 1456
# DYNAMIC
BYTES_IN_FRAME = (ADC_PARAMS['chirps'] * ADC_PARAMS['rx'] * ADC_PARAMS['tx'] *
ADC_PARAMS['IQ'] * ADC_PARAMS['samples'] *
ADC_PARAMS['bytes'])
BYTES_IN_FRAME_CLIPPED = (BYTES_IN_FRAME // BYTES_IN_PACKET) * BYTES_IN_PACKET
PACKETS_IN_FRAME = BYTES_IN_FRAME / BYTES_IN_PACKET
PACKETS_IN_FRAME_CLIPPED = BYTES_IN_FRAME // BYTES_IN_PACKET
UINT16_IN_PACKET = BYTES_IN_PACKET // 2
UINT16_IN_FRAME = BYTES_IN_FRAME // 2
# a = []
# b = []


class DCA1000:
"""Software interface to the DCA1000 EVM board via ethernet.

Attributes:
static_ip (str): IP to receive data from the FPGA
adc_ip (str): IP to send configuration commands to the FPGA
data_port (int): Port that the FPGA is using to send data
config_port (int): Port that the FPGA is using to read configuration commands from


General steps are as follows:
1. Power cycle DCA1000 and XWR1xxx sensor
2. Open mmWaveStudio and setup normally until tab SensorConfig or use lua script
3. Make sure to connect mmWaveStudio to the board via ethernet
4. Start streaming data
5. Read in frames using class

Examples:
>>> dca = DCA1000()
>>> adc_data = dca.read(timeout=.1)
>>> frame = dca.organize(adc_data, 128, 4, 256)

"""
def __init__(self,
configfilename,
static_ip='192.168.33.30',
adc_ip='192.168.33.180',
data_port=4098,
config_port=4096):
# Save network data
# self.static_ip = static_ip
# self.adc_ip = adc_ip
# self.data_port = data_port
# self.config_port = config_port

# Create configuration and data destinations
self.cfg_dest = (adc_ip, config_port)
self.cfg_recv = (static_ip, config_port)
self.data_recv = (static_ip, data_port)

# Create sockets
self.config_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM,
socket.IPPROTO_UDP)
self.data_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM,
socket.IPPROTO_UDP)

# Bind data socket to fpga
self.data_socket.bind(self.data_recv)

# Bind config socket to fpga
self.config_socket.bind(self.cfg_recv)

self.CLIport = serial.Serial('COM7', 115200)
self.Dataport = serial.Serial('COM8', 921600)

self.CfgFilename = configfilename

self.data = []
self.packet_count = []
self.byte_count = []

self.frame_buff = []

self.curr_buff = None
self.last_frame = None
self.lost_packets = None

def _serialConfig(self):
# Raspberry pi
# CLIport = serial.Serial('/dev/ttyACM0', 115200)
# Dataport = serial.Serial('/dev/ttyACM1', 921600)

# Windows
# self.CLIport = serial.Serial('COM11', 115200)
# self.Dataport = serial.Serial('COM12', 921600)

config = [line.rstrip('\r\n') for line in open(self.CfgFilename)]
for i in config:
if i == '' or i[0] == "#" or i[0] == "%":
continue
self.CLIport.write((i + '\n').encode())
print('sent->' + i)
time.sleep(0.1)
reply = self.CLIport.read(self.CLIport.in_waiting).decode()
print(reply)
return self.CLIport, self.Dataport

def _sensor_start(self):
self.CLIport.write("sensorStart\n".encode())
print("sent->" + "sensorStart")
time.sleep(0.1)
reply = self.CLIport.read(self.CLIport.in_waiting).decode()
print(reply)

def _sensor_stop(self):
self.CLIport.write("sensorStop\n".encode())
print("sent->" + "sensorStop")
time.sleep(0.1)
reply = self.CLIport.read(self.CLIport.in_waiting).decode()
print(reply)

def start(self):
# RECORD_START_CMD_CODE
# 5a a5 05 00 00 00 aa ee
print("->网口发送记录开始指令:\n{}".format(
self._send_command(CMD.RECORD_START_CMD_CODE)))
self._sensor_start()

print('commands send -> Start ')

def stop(self):
self._sensor_stop()
# RECORD_START_CMD_CODE
# 5a a5 06 00 00 00 aa ee
print("->网口发送记录终止指令:\n{}".format(
self._send_command(CMD.RECORD_STOP_CMD_CODE)))
self._close()

def _close(self):
# self.data_socket.close()
self.config_socket.close()
self.CLIport.close()
self.Dataport.close()

def configure(self):
# # SYSTEM_CONNECT_CMD_CODE
# # 5a a5 09 00 00 00 aa ee
# print("->连接确认:")
# print("{}".format(self.send_command(CMD.SYSTEM_CONNECT_CMD_CODE)))

# CONFIG_FPGA_GEN_CMD_CODE
# 5a a5 01 00 00 00 aa ee
print("->重置FPGA:")
print("{}".format(self._send_command(CMD.RESET_FPGA_CMD_CODE)))

# CONFIG_FPGA_GEN_CMD_CODE
# 5a a5 02 00 00 00 aa ee
print("->重置AR:")
print("{}".format(self._send_command(CMD.RESET_AR_DEV_CMD_CODE)))

# CONFIG_FPGA_GEN_CMD_CODE
# 5a a5 03 00 06 00 01 02 01 02 03 1e aa ee
print("->配置FPGA:")
print("{}".format(
self._send_command(CMD.CONFIG_FPGA_GEN_CMD_CODE, '0600',
'01020102031e')))

# CONFIG_PACKET_DATA_CMD_CODE
# 5a a5 0b 00 06 00 be 05 35 0c 00 00 aa ee
print("->配置PACKET:")
print("{}".format(
self._send_command(CMD.CONFIG_PACKET_DATA_CMD_CODE, '0600',
'c005350c0000')))

# # READ_FPGA_VERSION_CMD_CODE
# # 5a a5 0e 00 00 00 aa ee
# print("->读取FPGA版本:")
# print("{}".format(self.send_command_web(CMD.READ_FPGA_VERSION_CMD_CODE)))

def close(self):
"""Closes the sockets that are used for receiving and sending data

Returns:
None

"""
self.data_socket.close()
self.config_socket.close()

def read(self, timeout=1):
""" Read in a single packet via UDP

Args:
timeout (float): Time to wait for packet before moving on

Returns:
Full frame as array if successful, else None

"""
# Configure

self.data_socket.settimeout(timeout)

# Frame buffer
ret_frame = np.zeros(UINT16_IN_FRAME, dtype=np.int16)

# Wait for start of next frame
while True:
packet_num, byte_count, packet_data = self._read_data_packet()
# a.append(packet_num) # 获取包数组
# print(packet_num,byte_count)
if (packet_num - 1) % PACKETS_IN_FRAME_CLIPPED == 0:
packets_read = 1
ret_frame[0:UINT16_IN_PACKET] = packet_data
break

# Read in the rest of the frame
while True:
packet_num, byte_count, packet_data = self._read_data_packet()
# print(packet_num,byte_count)
# b.append(packet_num) # 获取包数组
packets_read += 1
if (packet_num - 1) % PACKETS_IN_FRAME_CLIPPED == 0:
self.lost_packets = PACKETS_IN_FRAME_CLIPPED - packets_read
# 返回-1 基本不丢包
# print("lost_packets:%d" %self.lost_packets)
return ret_frame

curr_idx = ((packet_num - 1) % PACKETS_IN_FRAME_CLIPPED)
try:
ret_frame[curr_idx * UINT16_IN_PACKET:(curr_idx + 1) *
UINT16_IN_PACKET] = packet_data
except:
pass

if packets_read > PACKETS_IN_FRAME_CLIPPED:
packets_read = 0

def _send_command(self, cmd, length='0000', body='', timeout=1):
"""Helper function to send a single commmand to the FPGA

Args:
cmd (CMD): Command code to send to the FPGA
length (str): Length of the body of the command (if any)
body (str): Body information of the command
timeout (int): Time in seconds to wait for socket data until timeout

Returns:
str: Response message

"""
# Create timeout exception
self.config_socket.settimeout(timeout)

# Create and send message
resp = ''
msg = codecs.decode(
''.join((CONFIG_HEADER, str(cmd), length, body, CONFIG_FOOTER)),
'hex')
try:
self.config_socket.sendto(msg, self.cfg_dest)
resp, addr = self.config_socket.recvfrom(MAX_PACKET_SIZE)
except socket.timeout as e:
print(e)
return resp

def _read_data_packet(self):
"""Helper function to read in a single ADC packet via UDP

Returns:

int: Current packet number, byte count of data that has already been read, raw ADC data in current packet
"""
data, addr = self.data_socket.recvfrom(MAX_PACKET_SIZE)
packet_num = struct.unpack('<1l', data[:4])[0]
byte_count = struct.unpack('>Q', b'\x00\x00' + data[4:10][::-1])[0]
packet_data = np.frombuffer(data[10:], dtype=np.int16)
return packet_num, byte_count, packet_data

def _listen_for_error(self):
"""Helper function to try and read in for an error message from the FPGA

Returns:
None

"""
self.config_socket.settimeout(None)
msg = self.config_socket.recvfrom(MAX_PACKET_SIZE)
if msg == b'5aa50a000300aaee':
print('stopped:', msg)

def _stop_stream(self):
"""Helper function to send the stop command to the FPGA

Returns:
str: Response Message

"""
return self._send_command(CMD.RECORD_STOP_CMD_CODE)

@staticmethod
def organize(raw_frame, num_chirps, num_rx, num_samples):
"""Reorganizes raw ADC data into a full frame

Args:
raw_frame (ndarray): Data to format
num_chirps: Number of chirps included in the frame
num_rx: Number of receivers used in the frame
num_samples: Number of ADC samples included in each chirp

Returns:
ndarray: Reformatted frame of raw data of shape (num_chirps, num_rx, num_samples)

"""

# ret = np.zeros(len(raw_frame) // 2, dtype=complex)

# # Separate IQ data
# ret[0::2] = raw_frame[0::4] + 1j * raw_frame[2::4]
# ret[1::2] = raw_frame[1::4] + 1j * raw_frame[3::4]
# return ret.reshape((num_chirps, num_rx, num_samples))


sampleI = np.zeros(len(raw_frame) // 2, dtype=complex)
sampleQ = np.zeros(len(raw_frame) // 2, dtype=complex)

sampleI[0::2] = raw_frame[0::4]
sampleI[1::2] = raw_frame[1::4]
sampleQ[0::2] = raw_frame[2::4]
sampleQ[1::2] = raw_frame[3::4]

sample = sampleQ + 1j * sampleI

return sample.reshape((num_chirps, num_rx, num_samples))

import mmwave.dsp as dsp
import mmwave.clustering as clu
from demo.visualizer import ellipse_visualize

if __name__ == "__main__":
DCA = DCA1000(configFileName)
DCA.configure()
time.sleep(3)
DCA._serialConfig()
DCA.start()
fig = plt.figure()

numADCSamples = 96
numTxAntennas = 3
numRxAntennas = 4
numLoopsPerFrame = 96
numChirpsPerFrame = numTxAntennas * numLoopsPerFrame

numRangeBins = numADCSamples
numDopplerBins = numLoopsPerFrame
numAngleBins = 64

range_resolution, bandwidth = dsp.range_resolution(numADCSamples)
doppler_resolution = dsp.doppler_resolution(bandwidth)

plotRangeDopp = False
plot2DscatterXY = True
plot2DscatterXZ = False
plot3Dscatter = False
plotCustomPlt = False

visTrigger = plot2DscatterXY + plot2DscatterXZ + plot3Dscatter + plotRangeDopp + plotCustomPlt
assert visTrigger < 2, "Can only choose to plot one type of plot at once"

singFrameView = False

# (1.5) Required Plot Declarations
# if plot2DscatterXY or plot2DscatterXZ:
# fig, axes = plt.subplots(1, 2)
# elif plot3Dscatter:
# fig = plt.figure()
# elif plotRangeDopp:
# fig = plt.figure()
# elif plotCustomPlt:
# print("Using Custom Plotting")

rx_index = [3,1,2,0,11,9,7,5,10,8,6,4];
sensor_number=12;
sensor_array = np.array([[1,1,0,0],[1,1,0,0],[1,1,1,1],[1,1,1,1]])

index = 0;
frame = np.zeros((100,96,12,96),dtype=complex)
while True:
try:
# (1) Reading in adc data
# start0 = time.time()
adc_data = DCA.read(timeout=.5)
# DCA.organize基本不占用时间
frame[index] = DCA.organize(adc_data, ADC_PARAMS['chirps'],ADC_PARAMS['rx'] * ADC_PARAMS['tx'],ADC_PARAMS['samples'])
if index == 99:
import scipy.io as io
mat_path = 'rx.mat'
io.savemat(mat_path, {'rx': frame})
else:
index = index + 1
# end0 = time.time()
# print('capture data is :%s seconds'%(end0 - start0))

except KeyboardInterrupt:
print("结束!")
DCA.stop()
plt.close()
break

plt.show()
exit(0)

前言

写在最前面, 如果博客内容与实际TI文档有任何出入,请以TI官方文档为准.

在上一篇博客中我们宏观的介绍了整个系统的硬件架构以及展示出了一副大的程序执行框图, 在这篇博客中我们将沿着系统软件执行流程, 走马观花式的介绍整个 mmWave Out of box demo 源码, 源码的代码极其的庞大, 同时这样一份系统性的代码阅读下来他的附加价值是很高的. 因为这篇博客介绍软件源码是走马观花式的, 所以强烈建议读者抽出时间仔细的研读源码, 对整个源码进行系统性的梳理以及细节的挖掘, 建议用时在15天左右. 这篇博客中我所给出的一些软件执行的框图也是在阅读源码的过程中进行整理的.

阅读思路

由于demo的源码非常的庞大, 我们需要对它进行宏观把握的前提下进行阅读, 在我进行源码结构梳理的时候我喜欢用一些显著的时间节点来进行分割, 比如 当我在CLI port进行配置时 demo的程序进行了如何的调用, 我在CLI port 发送 sensor start 的时候 demo程序中发生了什么, ARM端发送了什么样的信息告知DSP进行信号处理, DSP 如何将已经处理好的点云数据传送至ARM端等等. 同时在进行源码结构梳理的时候通过对源码进行功能划分, 是最直观且高效的结构梳理方法, Demo中的 功能区划主要有如下的几个:

  1. CLI: 负责与用户的信息交互 如配置, sensor的控制等
  2. MMWave: 与 DFE 进行控制信息交换等.
  3. DPC(DSP): Data process control 用于控制DSP端的信号处理和控制组件
  4. DPC(HWA): 用于控制arm端 hardware accelerator 的信号处理和控制组件
  5. DPM: data path manager, 顾名思义 就是在 MSS 和 DSS 中负责数据传输的管理组件
  6. Demo Misc: 负责 demo 中各各种为了使程序能够正常运行的中间组件比如协调控制, 信息交互等等.

在进行源码梳理的时候我们将围绕着 MSS 和 DSS 这两个相对独立的子系统上运行的程序展开, 同时在本篇博客中给出的软件执行框图中每一部软件执行的步骤都将使用在源码中的函数名进行表示.

下图展示了一个完整的代码结构整理, 他以上一章最后提到的系统执行流程为中心, 以 MSS和 DSS 两条线索分别展开.

源码走读

在代码执行初期 MSS 和 DSS 分别执行他们的初始化任务.

MSS初始化

DSS初始化

当MSS在串口收到Cfar配置信息或者Profile配置信息时


当MSS在串口收到sensor start指令时




当DSS从MSS那里得到DPM start的消息后

同时会向MSS端报告DPC_STARTED, MSS在得到DSS端DPC_STARTED的消息后会在CLI port打印 Done 的消息

之后DSS的HWA和DSS就会等待DFE的数据

当Frame Event的硬件中断到来, 表明当前的Frame已经开始了

在Frame结束后的后处理,如数据传输等

相关代码(plantuml)

为了方便 大家在自行整理源码结构, 以上软件流程框图的 plantuml 源码已经上传至github读者可以自行下载

https://github.com/Mr-Bulijiojio/NotesTIproject

前言

写在最前面, 如果博客内容与实际TI文档有任何出入,请以TI官方文档为准.

在前几个mmWave_Tutorial的博客中我们讲的关于TI out box demo, Ti 3D people counting demo 以及使用他们的数据进行的目标跟踪与航迹管理, 那读者们就很容易好奇在mmWave SoC 里面真正进行了怎样的处理,它的程序又是怎样编写的呢? 从本篇博客开始我们将一步一步的去了解 ti out of box demo 中的一些细节,我们将首先从整个软件的架构开始讲起同时也会结合相应的硬件结构进行讲述, 这个部分中如果读者有兴趣, 我强烈建议读者去自行阅读如下所示ti官方的文档

ti/mmwave_sdk_03_04_00_03/packages/ti/demo/xwr68xx/mmw/docs/doxygen/html/index.html

Hardware System Overview

TI out box demo运行在mmWave SoC上, 要去理解一个软件架构是如何工作的首先就需要去了解支持这个软件运行的硬件架构是怎样, 如下图

我们可以看到, 整个mmWave SoC系统由4个子系统构成

  1. RF/Analogy sub-system: 主要负责发射射频链路的信号合成, 移相以及功率放大, 接收射频链路的低噪声放大, 相干接收(混频,滤波)等等, 是TI MMIC的核心技术
  2. Radio processor sub-system: 主要负责后端的数字下变频(DDC), 相位矫正等等功能, 与射频前端的组合成一个由TI预programmed的blackbox, 也只对外部系统提供API
  3. Main sub-system: 由一个arm cortex R4F内核以及相应的数字系统构成(如总线, 硬件加速器(硬核), 数字外设等等), 主要负责系统的控制, 部分信号处理和与外部的交互等等
  4. DSP sub-system: 由一个C674x DSP大核和周边数字系统构成, 承担了系统大部分的信号处理工作, 同时也控制了由DFE来的数据通过LVDS向外传输的过程(DFE的数据会直接送入DSP的L3 Cache中).

mmWave雷达的控制和信号处理任务是分散在一个芯片上的各个不同子系统上的, 而我们在进行软件设计时往往需要设计一个统一的执行框架, 使得不同子系统的信息能够高效的交互, 那么ti的工程师在设计相关的软件时也始终贯彻这个理念. 对于我们这样的用户来说我们真正需要关心的是在ti进行芯片设计时预留出了哪些用户可以编程的部分, 答案当然是ARM和DSP, TI也提供了一套完整的系统执行框架-实时嵌入式操作系统SYSBIOS, 熟悉嵌入式操作系统的读者应该对于这个不会感到陌生, 而且ti官方也提供给我们了丰富的外设驱动程序, 只要在进行程序设计时, 能够正确的处理不同线程间的信号量, mmWave SoC系统开发人员就能够实现大部分的功能. 有了多个异构运算核心的统一的执行框架, 那么ARM和DSP之间信息如何交互呢, 在上图中我们可以看到Bus Matrix 和 Mailbox, 他们就是信息交互的硬件基础.

Software System Overview

下图显示了系统执行流程, 这幅图又大又长, 笔者在看到这幅图也是懵逼的但是我将在接下来的一篇博客中带着大家一起简单的过一遍这幅图. 在这里需要从这幅图中先粗略的get到一些上文中提到的概念的映射, 比如BSS代表RF/Analogy sub-system和Radio processor sub-system, MSS代表Main sub-system, BSS 代表 DSP sub-system. ARM和DSP即MSS 和 DSS 的线程间通信 IPC在图中的表示等等

前言

在前两个 tutorial 中我们通过编写脚本程序的方法得到了毫米波雷达输出的点云数据, 并进行了相关的数据可视化工作, 在这一篇博客当中我们将通过之前得到的毫米波雷达输出的点云, 进行人员的检测跟踪以及航迹的管理, 到这里读者可能会问了既然在ti官方的demo中已经给出了 gtrack相应的多目标跟踪算法的实现, 我们为什么又要重新写一遍呢? 这不是重复造轮子吗? 但其实对于处在学习阶段的我们来说, 这样重复造轮子的过程是必须的, 能够让我们快速的熟悉已有的一些成熟的算法, 并且在算法实现的过程中得到更深入的理解以便于我们后面进行更加深入的研究.

在本篇博客当中我们将基于已有的一个小的测试数据进行相关的目标跟踪及航迹管理的算法实现, 这篇博客将给出 matlab和Python 的两个实例. 目标跟踪及航迹管理的原理在这里我们就不进行介绍了, 可以参考目标跟踪有关博客的内容, 下图是在本篇博客中实现的目标跟踪及航迹管理算法的框图.

测试数据

测试数据的文件我将会尽快的放在个人博客上

测试数据参数

参数名 单位
数据每帧的扫描间隔T 1 s
距离单元格数 400
方位单元格数 200
距离分辨率 10 m
角度分辨率 0.5° °
过程噪声强度 0.01
信噪比SNR 18 dB
虚警概率 1e-5

第五帧测试数据如下图

这张图中我们可以看到在每一个距离单元与角度单元中都有数值, 那么我们需要进行CFAR处理进而得到检测点

CA-CFAR结果

我们将CFAR检测每一帧的结果叠加起来可以得到下图

通过CFAR检测后的点云数据就是我们进行目标跟踪及航迹管理的input

相关代码(Matlab)

CA-CFAR代码

一维CA-CFAR

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
function [Cfar] = CA_CFAR(Signal,PF,Refer,Pro)
%CA_CFAR
%Signal input signal
%PF False Probability
%Refer Reference Unit
%Pro Protected Unit

N_all = length(Signal);
L_slipper = 2*(Pro+Refer)+1; %处理窗的大小
Alpha = L_slipper*(PF^(-1/L_slipper)-1);%门限系数阿尔法

Cfar=zeros(1,N_all);
Signal = [zeros(1,Refer+Pro) Signal zeros(1,Refer+Pro)];
for i=1:N_all
Cfar(i)=Alpha*mean([Signal(i:i+Refer-1),Signal(i+Refer+2*Pro+1:i+2*Refer+2*Pro)]);
end`
end

二维CA-CFAR

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
function [CFAR] = CA_CFAR2D(Signal,PF,Refer,Pro)

SIZE = size(Signal);
L_slipper = 2*(Pro+Refer)+1; %处理窗的大小

temp=zeros(SIZE+[2*(Refer+Pro) 2*(Refer+Pro)]);
temp(Refer+Pro+1:Refer+Pro+SIZE(1),Refer+Pro+1:Refer+Pro+SIZE(2)) = Signal;

Window = ones(L_slipper,L_slipper);
Window(Refer+1:Refer+2*Pro+1,Refer+1:Refer+2*Pro+1) = zeros(2*Pro+1);
Av_Coff = L_slipper*L_slipper-(Pro*2+1)^2;

Alpha = Av_Coff*(PF^(-1/L_slipper^2)-1);%门限系数阿尔法

CFAR = zeros(SIZE);
for i=1:SIZE(1)
for j = 1:SIZE(2)
CFAR(i,j) = Alpha*sum(sum((temp(i:i+L_slipper-1,j:j+L_slipper-1).*Window).^2))/Av_Coff;
end
end
end

CA-CFAR使用示例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
clear all
close all
clc

load('Data_For_Test.mat');

result = cell(1,size(Data_For_Test.Raw_Data,2));

Range_Cell_Num = str2num(Data_For_Test.Range_Cell_Num);
Azimuth_Cell_Num = str2num(Data_For_Test.Azimuth_Cell_Num);
P_False_Alarm = str2num(Data_For_Test.P_False_Alarm);
Range_Resolution = 10;
Azimuth_Resolution = 0.5;

for i = 1:size(Data_For_Test.Raw_Data,2)

frame = Data_For_Test.Raw_Data{i};

PF = P_False_Alarm;
Refer = 8;
Pro = 1;

%% 1D cfar
% CFAR_thre = zeros(Range_Cell_Num,Azimuth_Cell_Num);
% for j = 1:Range_Cell_Num
% CFAR_thre(j,:) = CA_CFAR(frame(j,:),PF,Refer,Pro);
% end

%% 2D cfar
[CFAR_thre] = sqrt(CA_CFAR2D(frame,PF,Refer,Pro));

cfar_detect = (CFAR_thre<frame);
seq = [];
index = 1;
for k = 1:Range_Cell_Num
for j = 1:Azimuth_Cell_Num
if cfar_detect(k,j)
seq(index,:) = [k j];
index = index + 1;
end
end
end

seq = [ (seq(:,1)*Range_Resolution).*cos((seq(:,2)-100)*Azimuth_Resolution*pi/180) seq(:,1)*Range_Resolution.*sin((seq(:,2)-100)*Azimuth_Resolution*pi/180) ];

result{i} = seq;

end

figure
hold on
for i = 1:30
scatter(result{i}(:,1),result{i}(:,2));
end

figure
hold on
mesh(CFAR_thre)
mesh(frame)

卡尔曼滤波

1
2
3
4
5
6
7
8
9
10
11
function [x_est,P_k_k] = KalmanFilter(z_k,x_est_k_minus1,F,H,P_k,Q,R)
x_pre = F*x_est_k_minus1;
P = F*P_k*F' + Q;

z_pre = H*x_pre;
S = H*P*H' + R;
K = P*H'/(S);

x_est = x_pre + K*(z_k - z_pre);
P_k_k = (eye(size(K*H,1))-K*H)*P;
end

实验航迹和确认航迹类

实验航迹类(MN逻辑起始法)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
classdef MNstarter_KFCV
properties
seq = [];
M = 0;
N = 0;

% Kalman Coefficient
T = 1;
F;
H = [1 0 0 0;
0 1 0 0];
Gamma;
Q;
R;

x_pre = zeros(4,1);
P_k = eye(4);
z_pre = zeros(2,1);
P = eye(4);
S = eye(2);
K = zeros(4,2);

x_est = zeros(4,1);
end
methods
function obj = MNstarter_KFCV(Z,T)
obj.seq = Z;
obj.T = T;
obj.F = [1 0 T 0;
0 1 0 T;
0 0 1 0;
0 0 0 1];
obj.Gamma=[T*T/2 0;0 T*T/2;T 0;0 T];
obj.Q = obj.Gamma*obj.Gamma'*1;
obj.R = 1*obj.H*obj.H';
end
function obj = predict(obj)
obj.x_pre = obj.F*obj.seq(end,:)';
obj.P = obj.F*obj.P_k*obj.F' + obj.Q;

obj.z_pre = obj.H*obj.x_pre;
obj.S = obj.H*obj.P*obj.H' + obj.R;
obj.K = obj.P*obj.H'/(obj.S);
end
function obj = addseq(obj,Z)
obj.x_est = obj.x_pre + obj.K*(Z' - obj.z_pre);
obj.P_k = (eye(size(obj.K*obj.H,1))-obj.K*obj.H)*obj.P;
obj.seq = [obj.seq;obj.x_est'];
end
end
end

确认航迹类

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
classdef ConfirmTrajectory_KFCV
properties
seq = [];
scoreboard = 0;

% Kalman Coefficient
T = 1;
F;
H = [1 0 0 0;
0 1 0 0];
Gamma;
Q;
R;

x_pre = zeros(4,1);
P_k = eye(4);
z_pre = zeros(2,1);
P = eye(4);
S = eye(2);
K = zeros(4,2);

x_est = zeros(4,1);
end
methods
function obj = ConfirmTrajectory_KFCV(Z,T)
obj.seq = Z;
obj.T = T;
obj.F = [1 0 T 0;
0 1 0 T;
0 0 1 0;
0 0 0 1];
obj.Gamma=[T*T/2 0;0 T*T/2;T 0;0 T];
obj.Q = obj.Gamma*obj.Gamma'*1;
obj.R = 1*obj.H*obj.H';
end
function obj = predict(obj)
obj.x_pre = obj.F*obj.seq(end,:)';
obj.P = obj.F*obj.P_k*obj.F' + obj.Q;

obj.z_pre = obj.H*obj.x_pre;
obj.S = obj.H*obj.P*obj.H' + obj.R;
obj.K = obj.P*obj.H'/(obj.S);
end
function obj = addseq(obj,Z)
obj.x_est = obj.x_pre + obj.K*(Z' - obj.z_pre);
obj.P_k = (eye(size(obj.K*obj.H,1))-obj.K*obj.H)*obj.P;

% Staging data of 20 points
if size(obj.seq,1)<20
obj.seq = [obj.seq;obj.x_est'];
else
obj.seq = [obj.seq;obj.x_est'];
obj.seq(1,:) = [];
end

end
end
end

目标跟踪与航迹管理步进函数

matlab代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
function [Confirmed_trajectory] = Track_step_PNN(Measure)

persistent test_trajectory;
persistent confirmed_trajectory;
persistent Prev_Measure;
persistent initial_Flag;

if isempty(test_trajectory) && isempty(initial_Flag)
test_trajectory = {};
confirmed_trajectory = {};
Prev_Measure = [];
initial_Flag = 1;
end

% tracking parameter
DIS_THRE = chi2inv(0.99,2);
SCOREBOARD_THRE = 40;
N_PARAMETER = 6;
M_PARAMETER = 4;

V_MAX = 150;
V_MIN = 10;
T = 1;

Measure_temp = Measure;

% MN starter NN association
if ~isempty(Measure)
% associate with ConfirmedTrajectory
for kk = 1:size(confirmed_trajectory,2)
if ~isempty(Measure)
confirmed_trajectory(kk) = confirmed_trajectory(kk).predict();
temp = confirmed_trajectory(kk).z_pre'-Measure(:,1:2);
dis = diag(temp*inv(confirmed_trajectory(kk).S)*temp');%计算相邻两帧点迹距离

[val, pos] = min(dis);%选出距离最小的点
if val<DIS_THRE
confirmed_trajectory(kk) = confirmed_trajectory(kk).addseq(Measure(pos,:));
confirmed_trajectory(kk).scoreboard = 0;
Measure(pos,:) = [];
else
confirmed_trajectory(kk) = confirmed_trajectory(kk).addseq(confirmed_trajectory(kk).z_pre');
confirmed_trajectory(kk).scoreboard = confirmed_trajectory(kk).scoreboard + 1;
end
end
end
% terrible score delete ConfirmedTrajectory
delete_pos = [];
for kk = 1:size(confirmed_trajectory,2)
if confirmed_trajectory(kk).scoreboard > SCOREBOARD_THRE
delete_pos = [delete_pos kk];
end
end
confirmed_trajectory(delete_pos) = [];

% associate with MNstarter
for kk = 1:size(test_trajectory,2)
if ~isempty(Measure)
test_trajectory(kk) = test_trajectory(kk).predict();
temp = test_trajectory(kk).z_pre'-Measure(:,1:2);
dis = diag(temp*inv(test_trajectory(kk).S)*temp');%计算相邻两帧点迹距离

[val, pos] = min(dis);
if val<DIS_THRE
test_trajectory(kk) = test_trajectory(kk).addseq(Measure(pos,:));
test_trajectory(kk).N = test_trajectory(kk).N + 1;
test_trajectory(kk).M = test_trajectory(kk).M + 1;
Measure(pos,:) = [];
else
test_trajectory(kk) = test_trajectory(kk).addseq(test_trajectory(kk).z_pre');
test_trajectory(kk).N = test_trajectory(kk).N + 1;
end
end
end
% check whether satisfy MN logic
delete_pos = [];
for kk = 1:size(test_trajectory,2)
if test_trajectory(kk).N >= N_PARAMETER
if test_trajectory(kk).M >= M_PARAMETER
% transfer to ConfirmedTrajectory
confirmed_trajectory = [confirmed_trajectory TestToConfirmed(test_trajectory(kk),T)];
% free MNstarter
delete_pos = [delete_pos kk];
else
% delete MNstarter
delete_pos = [delete_pos kk];
end
end
end
test_trajectory(delete_pos) = [];

% directly start as a new MNstarter
test_trajectory = [test_trajectory directbegin(Measure,Prev_Measure,V_MIN,V_MIN,T)];
else
for kk = 1:size(confirmed_trajectory,2)
% predict
confirmed_trajectory(kk) = confirmed_trajectory(kk).predict();
% use prev point
confirmed_trajectory(kk) = confirmed_trajectory(kk).addseq(confirmed_trajectory(kk).z_pre');
% ConfirmedTrajectory scoreboard +1
confirmed_trajectory(kk).scoreboard = confirmed_trajectory(kk).scoreboard + 1;
end
% terrible score delete ConfirmedTrajectory
delete_pos = [];
for kk = 1:size(confirmed_trajectory,2)
if confirmed_trajectory(kk).scoreboard > SCOREBOARD_THRE
delete_pos = [delete_pos kk];
end
end
confirmed_trajectory(delete_pos) = [];

for kk = 1:size(test_trajectory,2)
% predict
test_trajectory(kk) = test_trajectory(kk).predict();
% use prev point
test_trajectory(kk) = test_trajectory(kk).addseq(test_trajectory(kk).z_pre');
% MNstarter N +1
test_trajectory(kk).N = test_trajectory(kk).N + 1;
end
% check whether satisfy MN logic
delete_pos = [];
for kk = 1:size(test_trajectory,2)
if test_trajectory(kk).N >= N_PARAMETER
if test_trajectory(kk).M >= M_PARAMETER
% transfer to ConfirmedTrajectory
confirmed_trajectory = [confirmed_trajectory TestToConfirmed(test_trajectory(kk),T)];
% free MNstarter
delete_pos = [delete_pos kk];
else
% delete MNstarter
delete_pos = [delete_pos kk];
end
end
end
test_trajectory(delete_pos) = [];
end

Prev_Measure = Measure_temp;

Confirmed_trajectory = confirmed_trajectory;
end

function [testroot] = directbegin(Measure,Prev_Measure,vmin,vmax,T)
testroot = MNstarter_KFCV.empty;

index = 1;
if ~isempty(Measure)
for i = 1:size(Prev_Measure,1)
% calculate distance between two frame
dis = [];
for j = 1:size(Measure,1)
dis = [dis norm(Prev_Measure(i,1:2)-Measure(j,1:2))];
end
[val pos] = min(dis);
if vmin*T<val<vmax*T
v = (Measure(pos,:)-Prev_Measure(i,:))/T;
testroot(index) = MNstarter_KFCV([[Prev_Measure(i,:) 0 0];[Measure(pos,:) v]],T);
index = index +1;
end
end
end

end

function [confirmedroot] = TestToConfirmed(testroot,T)
confirmedroot = ConfirmTrajectory_KFCV(testroot.seq,T);
confirmedroot.x_pre = testroot.x_pre;
confirmedroot.P_k = testroot.P_k;
confirmedroot.z_pre = testroot.z_pre;
confirmedroot.P = testroot.P;
confirmedroot.S = testroot.S;
confirmedroot.K = testroot.K;
confirmedroot.x_est = testroot.x_est;
end

python代码

二维跟踪

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
#
# File: 实现逻辑法航迹起始,NN法数据关联,卡尔曼滤波以及传统航迹管理(2维)
# 主要实现了MOT迭代函数
# Author: UESTC Yu Xuyao
# Email: yxy19991102@163.com
#

import scipy.io as scio
import numpy
import scipy.stats as st
import time
from IWR.track import *

# 2自由度卡方分布
threshold = st.chi2.ppf(0.80, 2)
T = 0.1

F = numpy.mat([[1, 0, T, 0], [0, 1, 0, T], [0, 0, 1, 0], [0, 0, 0, 1]])
H = numpy.mat([[1, 0, 0, 0], [0, 1, 0, 0]])
Gamma = numpy.mat([[T * T / 2, 0], [0, T * T / 2], [T, 0], [0, T]])
Q = Gamma * Gamma.T * 0.5
R = 0.1 * H * H.T

vmin = 0.2
vmax = 4

max_velocity = 1.5


def beginTrack(points):
global threshold, T, F, H, Gamma, Q, R, vmin, vmax

outside = []

index = 0
testroot = []

Z_k = (points[0])
Z_k_plus1 = (points[1])

for j in range(0, Z_k.shape[1]):
for k in range(0, Z_k_plus1.shape[1]):
d = max(0, numpy.linalg.norm(Z_k_plus1[:, k] - Z_k[:, j]) - vmax * T) + max(0, -numpy.linalg.norm(
Z_k_plus1[:, k] - Z_k[:, j]) + vmin * T)
D = d / numpy.linalg.det(R + R) * d
if D <= threshold:
temp = TestTrack(Z_k[:, j])
temp.addseq(Z_k_plus1[:, k])

Px0 = 5 * numpy.eye(4)
P = F * Px0 * F.T + Q
S = H * P * H.T + R * 0.2
K = P * H.T * numpy.linalg.inv(S)
Pkk = (numpy.eye(4) - K * H) * P

x_init = numpy.concatenate((Z_k_plus1[:, k], (Z_k_plus1[:, k] - Z_k[:, j]) / T), axis=0)
x_forest = F * x_init

temp.est = x_init
temp.pkk = Pkk
temp.x_predict = x_forest
temp.M = 2
temp.N = 2

testroot.append(temp)

return testroot


def TestToConfirmed(testroot):
confirmedroot = ConfirmedTrack(testroot.start)
confirmedroot.seq = testroot.seq
confirmedroot.est = testroot.est
confirmedroot.pkk = testroot.pkk
confirmedroot.seq = testroot.seq
confirmedroot.x_predict = testroot.x_predict
return confirmedroot


def MOT(Z_k, Z_k_prev, confirmedroot, testroot):
global threshold, T, F, H, Gamma, Q, R, vmin, vmax

Z_k_present = Z_k

if Z_k.shape[1] == 0:
if len(confirmedroot) != 0:
pos = []
for kk in range(0, len(confirmedroot)):
confirmedroot[kk].board += 1
if confirmedroot[kk].board >= 4:
pos.append(kk)
confirmedroot = numpy.delete(confirmedroot, pos)
confirmedroot = confirmedroot.tolist()

if Z_k.shape[1] != 0:
if len(confirmedroot) != 0:
pos = []
for kk in range(0, len(confirmedroot)):
if Z_k.shape[1] == 0:
break

P = F * confirmedroot[kk].pkk * F.T + Q
S = H * P * H.T + R
K = P * H.T * numpy.linalg.inv(S)
outside = H * confirmedroot[kk].x_predict
V = []
for i in range(0, Z_k.shape[1]):
V.append((Z_k[:, i] - outside).T * numpy.linalg.inv(S) * (Z_k[:, i] - outside))

index = numpy.argmin(V)
val = numpy.amin(V)
if val <= threshold:
confirmedroot[kk].addseq(Z_k[:, index])
est = confirmedroot[kk].x_predict + K * (Z_k[:, index] - outside)
confirmedroot[kk].addest(est)
confirmedroot[kk].pkk = (numpy.eye(4) - K * H) * P
confirmedroot[kk].x_predict = F * est
confirmedroot[kk].board = 0

Z_k = numpy.delete(Z_k, index, axis=1)
else:
confirmedroot[kk].addseq(outside)
confirmedroot[kk].addest(confirmedroot[kk].x_predict)
confirmedroot[kk].pkk = P
confirmedroot[kk].x_predict = F * confirmedroot[kk].est[:, -1]
confirmedroot[kk].board += 1

if confirmedroot[kk].board >= 4 or numpy.linalg.norm(confirmedroot[kk].est[2:4, -1]) > max_velocity:
pos.append(kk)

confirmedroot = numpy.delete(confirmedroot, pos)
confirmedroot = confirmedroot.tolist()

if Z_k.shape[1] != 0:
if len(testroot) != 0:
pos = []
for kk in range(0, len(testroot)):
if Z_k.shape[1] == 0:
break

P = F * testroot[kk].pkk * F.T + Q
S = H * P * H.T + R
K = P * H.T * numpy.linalg.inv(S)
outside = H * testroot[kk].x_predict
V = []
for i in range(0, Z_k.shape[1]):
V.append((Z_k[:, i] - outside).T * numpy.linalg.inv(S) * (Z_k[:, i] - outside))

index = numpy.argmin(V)
val = numpy.amin(V)
if val <= threshold:
testroot[kk].addseq(Z_k[:, index])
est = testroot[kk].x_predict + K * (Z_k[:, index] - outside)
testroot[kk].addest(est)
testroot[kk].pkk = (numpy.eye(4) - K * H) * P
testroot[kk].x_predict = F * est
testroot[kk].M += 1
testroot[kk].N += 1
Z_k = numpy.delete(Z_k, index, axis=1)
else:
testroot[kk].addseq(outside)
testroot[kk].addest(testroot[kk].x_predict)
testroot[kk].pkk = P
testroot[kk].x_predict = F * testroot[kk].est[:, -1]
testroot[kk].N += 1

if testroot[kk].N == 6:
if testroot[kk].M >= 4 and numpy.linalg.norm(testroot[kk].est[2:4, -1]) < max_velocity:
confirmedroot.append(TestToConfirmed(testroot[kk]))
else:
pos.append(kk)

testroot = numpy.delete(testroot, pos)
testroot = testroot.tolist()

if Z_k.shape[1] != 0:
testroot.extend(beginTrack([Z_k_prev, Z_k_present]))

Z_k_prev = Z_k_present
return confirmedroot, testroot, Z_k_present

三维跟踪

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
#
# File: 实现逻辑法航迹起始,NN法数据关联,卡尔曼滤波以及传统航迹管理(3维)
# 主要实现了MOT迭代函数
# Author: UESTC YuXuyao
# Email: yxy19991102@163.com
#

import scipy.io as scio
import numpy
import scipy.stats as st
import time
from IWR.track import *

threshold = st.chi2.ppf(0.80, 3)
T = 0.1

F = numpy.mat([[1, 0, 0, T, 0, 0], [0, 1, 0, 0, T, 0], [0, 0, 1, 0, 0, T], [0, 0, 0, 1, 0, 0], [0, 0, 0, 0, 1, 0],
[0, 0, 0, 0, 0, 1]])
H = numpy.mat([[1, 0, 0, 0, 0, 0], [0, 1, 0, 0, 0, 0], [0, 0, 1, 0, 0, 0]])
Gamma = numpy.mat([[T * T / 2, 0, 0], [0, T * T / 2, 0], [0, 0, T * T / 2], [T, 0, 0], [0, T, 0], [0, 0, T]])
Q = Gamma * Gamma.T * 0.5
R = 0.1 * H * H.T

vmin = 0.2
vmax = 4

max_velocity = 2

def beginTrack(points):
global threshold, T, F, H, Gamma, Q, R, vmin, vmax

outside = []

index = 0
testroot = []

Z_k = (points[0])
Z_k_plus1 = (points[1])

for j in range(0, Z_k.shape[1]):
for k in range(0, Z_k_plus1.shape[1]):
d = max(0, numpy.linalg.norm(Z_k_plus1[:, k] - Z_k[:, j]) - vmax * T) + max(0, -numpy.linalg.norm(
Z_k_plus1[:, k] - Z_k[:, j]) + vmin * T)
D = d / numpy.linalg.det(R + R) * d
if D <= threshold:
temp = TestTrack(Z_k[:, j])
temp.addseq(Z_k_plus1[:, k])

Px0 = 5 * numpy.eye(6)
P = F * Px0 * F.T + Q
S = H * P * H.T + R * 0.2
K = P * H.T * numpy.linalg.inv(S)
Pkk = (numpy.eye(6) - K * H) * P

x_init = numpy.concatenate((Z_k_plus1[:, k], (Z_k_plus1[:, k] - Z_k[:, j]) / T), axis=0)
x_forest = F * x_init

temp.est = x_init
temp.pkk = Pkk
temp.x_predict = x_forest
temp.M = 2
temp.N = 2

testroot.append(temp)

return testroot


def TestToConfirmed(testroot):
confirmedroot = ConfirmedTrack(testroot.start)
confirmedroot.seq = testroot.seq
confirmedroot.est = testroot.est
confirmedroot.pkk = testroot.pkk
confirmedroot.seq = testroot.seq
confirmedroot.x_predict = testroot.x_predict
return confirmedroot


def MOT(Z_k, Z_k_prev, confirmedroot, testroot):
global threshold, T, F, H, Gamma, Q, R, vmin, vmax

Z_k_present = Z_k

if Z_k.shape[1] == 0:
if len(confirmedroot) != 0:
pos = []
for kk in range(0, len(confirmedroot)):
confirmedroot[kk].board += 1
if confirmedroot[kk].board >= 4:
pos.append(kk)
confirmedroot = numpy.delete(confirmedroot, pos)
confirmedroot = confirmedroot.tolist()

if Z_k.shape[1] != 0:
if len(confirmedroot) != 0:
pos = []
for kk in range(0, len(confirmedroot)):
if Z_k.shape[1] == 0:
break

P = F * confirmedroot[kk].pkk * F.T + Q
S = H * P * H.T + R
K = P * H.T * numpy.linalg.inv(S)
outside = H * confirmedroot[kk].x_predict
V = []
for i in range(0, Z_k.shape[1]):
V.append((Z_k[:, i] - outside).T * numpy.linalg.inv(S) * (Z_k[:, i] - outside))

index = numpy.argmin(V)
val = numpy.amin(V)
if val <= threshold:
confirmedroot[kk].addseq(Z_k[:, index])
est = confirmedroot[kk].x_predict + K * (Z_k[:, index] - outside)
confirmedroot[kk].addest(est)
confirmedroot[kk].pkk = (numpy.eye(6) - K * H) * P
confirmedroot[kk].x_predict = F * est
confirmedroot[kk].board = 0

Z_k = numpy.delete(Z_k, index, axis=1)
else:
confirmedroot[kk].addseq(outside)
confirmedroot[kk].addest(confirmedroot[kk].x_predict)
confirmedroot[kk].pkk = P
confirmedroot[kk].x_predict = F * confirmedroot[kk].est[:, -1]
confirmedroot[kk].board += 1

if confirmedroot[kk].board >= 4 or numpy.linalg.norm(confirmedroot[kk].est[3:6, -1]) > max_velocity:
pos.append(kk)

confirmedroot = numpy.delete(confirmedroot, pos)
confirmedroot = confirmedroot.tolist()

if Z_k.shape[1] != 0:
if len(testroot) != 0:
pos = []
for kk in range(0, len(testroot)):
if Z_k.shape[1] == 0:
break

P = F * testroot[kk].pkk * F.T + Q
S = H * P * H.T + R
K = P * H.T * numpy.linalg.inv(S)
outside = H * testroot[kk].x_predict
V = []
for i in range(0, Z_k.shape[1]):
V.append((Z_k[:, i] - outside).T * numpy.linalg.inv(S) * (Z_k[:, i] - outside))

index = numpy.argmin(V)
val = numpy.amin(V)
if val <= threshold:
testroot[kk].addseq(Z_k[:, index])
est = testroot[kk].x_predict + K * (Z_k[:, index] - outside)
testroot[kk].addest(est)
testroot[kk].pkk = (numpy.eye(6) - K * H) * P
testroot[kk].x_predict = F * est
testroot[kk].M += 1
testroot[kk].N += 1
Z_k = numpy.delete(Z_k, index, axis=1)
else:
testroot[kk].addseq(outside)
testroot[kk].addest(testroot[kk].x_predict)
testroot[kk].pkk = P
testroot[kk].x_predict = F * testroot[kk].est[:, -1]
testroot[kk].N += 1

if testroot[kk].N == 6:
if testroot[kk].M >= 4 and numpy.linalg.norm(testroot[kk].est[3:6, -1]) < max_velocity:
confirmedroot.append(TestToConfirmed(testroot[kk]))
else:
pos.append(kk)

testroot = numpy.delete(testroot, pos)
testroot = testroot.tolist()

if Z_k.shape[1] != 0:
testroot.extend(beginTrack([Z_k_prev, Z_k_present]))

Z_k_prev = Z_k_present
return confirmedroot, testroot, Z_k_present

目标跟踪与航迹管理实例脚本

matlab示例

其中 measure.mat 是由CFAR检测得到的每一帧检测点的集合

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
clear all
close all
clc

load('measure.mat');
figure
hold on

for i = 1:size(measure,2)

hold off
plot(1,1)
axis([0 4000 -4000 4000]);
hold on
[Confirmed_trajectory] = Track_step_PNN(measure{i});

for j = 1:size(Confirmed_trajectory,2)
plot(Confirmed_trajectory(j).seq(:,1),Confirmed_trajectory(j).seq(:,2),'-o','linewidth',1);
axis([0 4000 -4000 4000]);
end

pause(0.1);
end

python示例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
#
# File: 读取IWR6843处理得到的点云数据(3维)进行目标跟踪,同时调用openGL对第一个航迹进行显示(多个显示未完成)
# Author: UESTC Yu Xuyao
# Email: yxy19991102@163.com
#

import serial
import time
import numpy as np
import pyqtgraph as pg
from pyqtgraph.Qt import QtGui
import pyqtgraph.opengl as gl
from IWR.MOT3D import *
import numpy
from sklearn.cluster import dbscan
import pandas

# Change the configuration file name
configFileName = 'profile_iwr6843_ods_3d.cfg'

CLIport = {}
Dataport = {}
byteBuffer = np.zeros(2 ** 15, dtype='uint8')
byteBufferLength = 0

testroot = []
confirmedroot = []
Z_k_prev = numpy.mat([])


# ------------------------------------------------------------------
# Function to configure the serial ports and send the data from
# the configuration file to the radar
def serialConfig(configFileName):
global CLIport
global Dataport
# Open the serial ports for the configuration and the data ports

# Linux
# CLIport = serial.Serial('/dev/ttyACM0', 115200)
# Dataport = serial.Serial('/dev/ttyACM1', 921600)

# Windows
CLIport = serial.Serial('COM7', 115200)
Dataport = serial.Serial('COM8', 921600)

# Read the configuration file and send it to the board
config = [line.rstrip('\r\n') for line in open(configFileName)]
for i in config:
print(i)
CLIport.write((i + '\n').encode())
# wait for reply
time.sleep(0.1)
reply = CLIport.read(CLIport.in_waiting).decode()
print(reply)

return CLIport, Dataport

# ------------------------------------------------------------------

# Funtion to read and parse the incoming data
def readAndParseData(Dataport):
global byteBuffer, byteBufferLength

# Constants
OBJ_STRUCT_SIZE_BYTES = 12
BYTE_VEC_ACC_MAX_SIZE = 2 ** 15
MMWDEMO_UART_MSG_DETECTED_POINTS = 1
MMWDEMO_UART_MSG_RANGE_PROFILE = 2
maxBufferSize = 2 ** 15
tlvHeaderLengthInBytes = 8
pointLengthInBytes = 16
magicWord = [2, 1, 4, 3, 6, 5, 8, 7]

# Initialize variables
magicOK = 0 # Checks if magic number has been read
dataOK = 0 # Checks if the data has been read correctly
frameNumber = 0
detObj = {}

readBuffer = Dataport.read(Dataport.in_waiting)
byteVec = np.frombuffer(readBuffer, dtype='uint8')
byteCount = len(byteVec)

# Check that the buffer is not full, and then add the data to the buffer
if (byteBufferLength + byteCount) < maxBufferSize:
byteBuffer[byteBufferLength:(byteBufferLength + byteCount)] = byteVec[0:byteCount]
byteBufferLength = byteBufferLength + byteCount

# Check that the buffer has some data
if byteBufferLength > 16:

# Check for all possible locations of the magic word
possibleLocs = np.where(byteBuffer == magicWord[0])[0]

# Confirm that is the beginning of the magic word and store the index in startIdx
startIdx = []
for loc in possibleLocs:
check = byteBuffer[loc:loc + 8]
if np.all(check == magicWord):
startIdx.append(loc)

# Check that startIdx is not empty
if startIdx:

# Remove the data before the first start index
if 0 < startIdx[0] < byteBufferLength:
byteBuffer[:byteBufferLength - startIdx[0]] = byteBuffer[startIdx[0]:byteBufferLength]
byteBuffer[byteBufferLength - startIdx[0]:] = np.zeros(len(byteBuffer[byteBufferLength - startIdx[0]:]),
dtype='uint8')
byteBufferLength = byteBufferLength - startIdx[0]

# Check that there have no errors with the byte buffer length
if byteBufferLength < 0:
byteBufferLength = 0

# Read the total packet length
totalPacketLen = int.from_bytes(byteBuffer[12:12 + 4], byteorder='little')

# Check that all the packet has been read
if (byteBufferLength >= totalPacketLen) and (byteBufferLength != 0):
magicOK = 1

# If magicOK is equal to 1 then process the message
if magicOK:

# Initialize the pointer index
idX = 0

# Read the header
magicNumber = byteBuffer[idX:idX + 8]
idX += 8
version = format(int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little'), 'x')
idX += 4
totalPacketLen = int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little')
idX += 4
platform = format(int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little'), 'x')
idX += 4
frameNumber = int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little')
idX += 4
timeCpuCycles = int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little')
idX += 4
numDetectedObj = int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little')
idX += 4
numTLVs = int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little')
idX += 4
subFrameNumber = int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little')
idX += 4

# Read the TLV messages
for tlvIdx in range(numTLVs):

# Check the header of the TLV message
tlv_type = int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little')
idX += 4
tlv_length = int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little')
idX += 4

# Read the data depending on the TLV message
if tlv_type == MMWDEMO_UART_MSG_DETECTED_POINTS:

# Initialize the arrays
x = np.zeros(numDetectedObj, dtype=np.float32)
y = np.zeros(numDetectedObj, dtype=np.float32)
z = np.zeros(numDetectedObj, dtype=np.float32)
velocity = np.zeros(numDetectedObj, dtype=np.float32)

for objectNum in range(numDetectedObj):
# Read the data for each object
x[objectNum] = byteBuffer[idX:idX + 4].view(dtype=np.float32)
idX += 4
y[objectNum] = byteBuffer[idX:idX + 4].view(dtype=np.float32)
idX += 4
z[objectNum] = byteBuffer[idX:idX + 4].view(dtype=np.float32)
idX += 4
velocity[objectNum] = byteBuffer[idX:idX + 4].view(dtype=np.float32)
idX += 4

# Store the data in the detObj dictionary
detObj = {"numObj": numDetectedObj, "x": x, "y": y, "z": z, "velocity": velocity}
dataOK = 1

# Remove already processed data
if 0 < idX < byteBufferLength:
shiftSize = totalPacketLen

byteBuffer[:byteBufferLength - shiftSize] = byteBuffer[shiftSize:byteBufferLength]
byteBuffer[byteBufferLength - shiftSize:] = np.zeros(len(byteBuffer[byteBufferLength - shiftSize:]),
dtype='uint8')
byteBufferLength = byteBufferLength - shiftSize

# Check that there are no errors with the buffer length
if byteBufferLength < 0:
byteBufferLength = 0

return dataOK, frameNumber, detObj


# ------------------------------------------------------------------

# Funtion to update the data and display in the plot
def update():
dataOk = 0
global detObj, confirmedroot, testroot, Z_k_prev, LinePlot
x = []
y = []
z = []
# Read and parse the received data
dataOk, frameNumber, detObj = readAndParseData(Dataport)

if dataOk:
x = detObj["x"]
y = detObj["y"]
z = detObj["z"]

if len(x) != 0:
core_samples, cluster_ids = dbscan(np.vstack((x, y, z)).T, eps=0.3, min_samples=3)
df = pandas.DataFrame(np.c_[np.vstack((x, y, z)).T, cluster_ids],
columns=['x', 'y', 'z', 'cluster_id'])
df = df[df.cluster_id != -1]
count = df['cluster_id'].value_counts()

x = []
y = []
z = []
if count.shape[0] >= 1:
for i in range(0, count.shape[0]):
df1 = df[df.cluster_id == i]
mean = np.mean([df1['x'], df1['y'], df1['z']], 1)
x.append(mean[0])
y.append(mean[1])
z.append(mean[2])

Z_k = numpy.mat([x, y, z])

confirmedroot, testroot, Z_k_prev = MOT(Z_k, Z_k_prev, confirmedroot, testroot)

if len(confirmedroot) > 0:
tmppos = confirmedroot[0].est[0:3, :].T.A
for i in range(1,len(confirmedroot)):
tmppos = numpy.vstack((tmppos,confirmedroot[i].est[0:3, :].T.A))
LinePlot.setData(pos=tmppos)
QtGui.QApplication.processEvents()
else:
LinePlot.setData(pos=np.array([100, 100, 100]))
QtGui.QApplication.processEvents()

return dataOk


# ------------------------- MAIN -----------------------------------------

# Configurate the serial port
CLIport, Dataport = serialConfig(configFileName)

# START QtAPPfor the plot
app = QtGui.QApplication([])

app = pg.mkQApp()

view = gl.GLViewWidget()
view.setBackgroundColor('k')
view.show()

axis = gl.GLAxisItem()
axis.scale(10, 10, 10)

view.addItem(axis)

LinePlot = gl.GLScatterPlotItem()
view.addItem(LinePlot)

# Main loop
detObj = {}
frameData = {}
currentIndex = 0
while True:
try:
# Update the data and check if the data is okay
dataOk = update()

if dataOk:
# Store the current frame into frameData
frameData[currentIndex] = detObj
currentIndex += 0

time.sleep(0.01) # Sampling frequency of 20 Hz

# Stop the program and close everything if Ctrl + c is pressed
except KeyboardInterrupt:
CLIport.write(('sensorStop\n').encode())
print('sensorStop\n')
CLIport.close()
Dataport.close()
win.close()
break

前言

在上一篇博客中mmWave_Tutorial1我们讲述了关于Ti官方demo数据读取以及数据使用并给出了一段实例程序, 细心的读者在使用完历程后会发现ti官方demo所输出的点云数据非常的稀疏, 这一部分是由于mmwave设备天线的稀疏, 一部分是由于在ti官方demo中, 角度估计使用的是传统波束形成的方法其在天线维进行fft来进行角度估计, 而这样稀疏的点云数据不利于我们进行后续更高阶的应用, 而在TI IndustrialToolbox 的 3D people counting demo 中使用了 Capon自适应波束形成算法, 能够生成相对密集的点云, 为我们后续进行聚类, 目标跟踪行为识别等算法的实现提供了基础. 程序的源码, 预编译二进制文件以及相应的说明文档可以在如下的目录中找到.

\ti\mmwave_industrial_toolbox_4_5_1\labs\people_counting\68xx_3D_people_counting

需要注意的是在这篇博客当中所使用到的硬件设备为IWR6843ISK,IWR8843ISK-AOP或者IWR6843ISK-ODS. 同样IWR1843Boost也可以使用

在这篇博客中我们利用之前的方法来对TI IndustrialToolbox 中的 3D people counting demo 进行上位机程序的编写, 来实现数据的解析与读取.

3D people counting demo概述

与ti out of box demo相比 3D people counting demo实现了如下两个新的功能:

  1. 雷达前端信号处理链( 包含Capon自适应波数形成算法)
  2. gtrack多目标跟踪算法

3D people counting demo的框图如下图所示

需要注意的是gtrack多目标跟踪算法是实现在ARM上的

同时由于xWRxx43的雷达SoC采用了3发4收的天线通道, 与iwr1642相比, 收获了在俯仰角(elevation)上的分辨能力而不单单仅限于方位角(azimuth)上的分辨能力, 而不同的天线布局会影响到后级的信号处理, 在本篇博客中我不会去详细的介绍信号处理我将会将重点放在如何读取从雷达返回的已经处理好的点云数据.

数据传输以及格式

CLI port 数据传输及格式

在之前讲述ti out of box demo 我们提到了 CLI port 是用来发送对demo程序的配置信息, 而在 3D people counting demo中, 不同于 ti out of box demo的任务, 3D people counting demo需要进行天线阵列排布, 房间边界, 状态参数等等的配置, 所以它的 CLI port数据传输的格式与相应的意义与ti out of box demo 就有所不同

具体的数据传输格式以及相应的含义可以在如下文档中的Modifying Configuration File的章节中找到

/ti/mmwave_industrial_toolbox_4_5_1/labs/people_counting/68xx_3D_people_counting/docs/3d_pplcount_user_guide.pdf

Data port 数据传输及格式

与在ti out of box demo相似, 3D people counting 在数据传输时使用了 类似的 TLV 格式, 如下图所示

与ti out of box demo 不同的是,TLV的定义与数据传输的格式, 具体的数据传输格式可以在如下文档中的 Data Formats 的章节中找到

/ti/mmwave_industrial_toolbox_4_5_1/labs/people_counting/68xx_3D_people_counting/docs/3d_pplcount_user_guide.pdf

3D people counting user Guide中我们可以看到 3D people counting 的 TLV 有如下三种,分别是

  1. Point Cloud TLV
  2. Target Object TLV
  3. Target Index TLV

Point Cloud TLV用于传输雷达信号处理得到的点云数据, Target Object TLV用于输出gtrack算法输出的 target 的状态 比如位置, 速度, 加速度等, Target Index TLV 用于输出每个target 的独立的编号.
它的数据定义也是通过结构体来实现的, 我们可以参考ti out of box demo 从有关的定义来进行驱动程序的编写

相关代码(Matlab)

注意: 为了方便读者的使用, 我已经将3D people counting demo 中进行了修改, 删除了其中目标跟踪的部分并编译成了二进制文件, 需要请联系我

这部分的代码与 ti out of box demo 类似, 但是为了方便读者们复制粘贴使用 使用, 在这里依然给出相应的各个功能部分的代码, Data port 数据传输及格式的不同带来的变化主要反映在函数 function [dataOk, frameNumber, detObj] = readAndParseData(DATA_sphandle)

打开传输串口

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
% Description: 
% Author: Yu Xuyao
% Email: yxy19991102@163.com
% Date: 2020-12-25 00:26:00
% LastEditTime: 2020-12-25 00:26:00
% LastEditors: Yu Xuyao

function [CLI_sphandle,DATA_sphandle] = setupUART(CLIcomPortString,DATAcomPortString)

if strcmp(CLIcomPortString(1:3),'COM') || strcmp(DATAcomPortString(1:3),'COM')

CLI_sphandle = serial(CLIcomPortString,'BaudRate',115200);
set(CLI_sphandle,'Parity','none')
set(CLI_sphandle,'Terminator','LF')
fopen(CLI_sphandle);

DATA_sphandle = serial(DATAcomPortString,'BaudRate',921600);
set(DATA_sphandle,'Terminator', '');
set(DATA_sphandle,'InputBufferSize', 65536);
set(DATA_sphandle,'Timeout',10);
set(DATA_sphandle,'ErrorFcn',@dispError);
set(DATA_sphandle,'BytesAvailableFcnMode','byte');
set(DATA_sphandle,'BytesAvailableFcnCount', 2^16+1);%BYTES_AVAILABLE_FCN_CNT);
set(DATA_sphandle,'BytesAvailableFcn',@readUartCallbackFcn);
fopen(DATA_sphandle);
else
fprintf('COM port string format wrong\n');
end
end

CLI port配置数据

这个 CLI port 的配置数据是用来配置IWR8843ISK-AOP, 其他的型号的mmWave雷达开发板的有关配置可以在如下的文件中找到

\ti\mmwave_industrial_toolbox_4_5_1\labs\people_counting\68xx_3D_people_counting\chirp_configs

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
sensorStop
flushCfg
dfeDataOutputMode 1
channelCfg 15 7 0
adcCfg 2 1
adcbufCfg -1 0 1 1 1
lowPower 0 0
profileCfg 0 60.75 7 7 41.10 0 0 54.71 1 96 2950.00 0 0 36
chirpCfg 0 0 0 0 0 0 0 1
chirpCfg 1 1 0 0 0 0 0 2
chirpCfg 2 2 0 0 0 0 0 4
frameCfg 0 2 96 0 50 1 0
dynamicRACfarCfg -1 4 4 2 4 8 16 4 4 4.00 4.50 0.50 1 1
staticRACfarCfg -1 4 4 2 4 8 16 4 6 12.00 13.00 0.30 0 0
dynamicRangeAngleCfg -1 0.75 0.0010 1 0
dynamic2DAngleCfg -1 1.5 0.0300 1 0 1 0.50 0.85 8.00
staticRangeAngleCfg -1 1 8 4
antGeometry0 -1 -1 0 0 -3 -3 -2 -2 -1 -1 0 0
antGeometry1 -1 0 -1 0 -3 -2 -3 -2 -3 -2 -3 -2
antPhaseRot 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
fovCfg -1 70.0 20.0
compRangeBiasAndRxChanPhase 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0
staticBoundaryBox -2 2 2 8.5 -2 2
boundaryBox -2.5 2.5 0.5 9 -2.5 2.5
sensorPosition 2 0 0
gatingParam 3 2 2 2 0
stateParam 3 3 10 40 5 600
allocationParam 00 800 0.1 15 0.1 20
trackingCfg 1 2 1000 20 67 105 50
sensorStart

读取CLI port配置数据

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
% Description: 
% Author: Yu Xuyao
% Email: yxy19991102@163.com
% Date: 2020-12-25 00:26:00
% LastEditTime: 2020-12-25 00:26:00
% LastEditors: Yu Xuyao

function [configcmd] = readCfgFile(configFile)

configcmd = cell(1,100);
fid = fopen(configFile, 'r');
if fid == -1
fprintf('File %s not found!\n', configFile);
return;
else
fprintf('Opening configuration file %s ...\n', configFile);
end
tline = fgetl(fid);
k=1;
while ischar(tline)
if ~isempty(tline)
configcmd{k} = tline;
k = k + 1;
end
tline = fgetl(fid);
end
configcmd = configcmd(1:k-1);
fclose(fid);
end

发送CLI port的配置数据

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
% Description: 
% Author: Yu Xuyao
% Email: yxy19991102@163.com
% Date: 2020-12-25 00:26:00
% LastEditTime: 2020-12-25 00:26:00
% LastEditors: Yu Xuyao

function sendCfg(UART_sphandle,configcmd)
mmwDemoCliPrompt = char('mmwDemo:/>');

%Send CLI configuration to IWR
fprintf('Sending configuration to IWR radar\n');
for k=1:length(configcmd)
command = configcmd{k};
fprintf(UART_sphandle, command);
fprintf('%s\n', command);
echo = fgetl(UART_sphandle); % Get an echo of a command
done = fgetl(UART_sphandle); % Get "Done"
fprintf('%s\n', done);
prompt = fread(UART_sphandle, size(mmwDemoCliPrompt,2)); % Get the prompt back
fprintf('%s\n', prompt);
end
end

读取Data port数据

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
% Description: 
% Author: Yu Xuyao
% Email: yxy19991102@163.com
% Date: 2020-12-25 00:26:00
% LastEditTime: 2020-12-25 00:26:00
% LastEditors: Yu Xuyao

function [dataOk, frameNumber, pointStruct,targetStruct2D] = readAndParseData(DATA_sphandle)
pointStruct_SIZE_BYTES = 8;
targetStruct2D_SIZE_BYTES = 40;
BYTE_VEC_ACC_MAX_SIZE = 2^16;
POINT_CLOUD_2D = 6;
TARGET_LIST_3D = 7;
TARGET_INDEX = 8;
maxBufferSize = BYTE_VEC_ACC_MAX_SIZE;

pointStruct =[];
targetStruct2D = [];
frameNumber = 0;

persistent byteBuffer
if isempty(byteBuffer)
byteBuffer = zeros(maxBufferSize,1);
end

persistent byteBufferLength
if isempty(byteBufferLength)
byteBufferLength = 0;
end

persistent magiNotOkCounter
if isempty(magiNotOkCounter)
magiNotOkCounter = 0;
end

magicOk = 0;
dataOk = 0;

bytesToRead = get(DATA_sphandle,'BytesAvailable');
if (bytesToRead ~= 0)
% Read the Data Serial Port
[bytevec, byteCount] = fread(DATA_sphandle, bytesToRead, 'uint8');

% Check if the buffer is not full, and then add the data to the buffer:
if(byteBufferLength + byteCount < maxBufferSize)
byteBuffer(byteBufferLength+1:byteBufferLength + byteCount) = bytevec(1:byteCount);
byteBufferLength = byteBufferLength + byteCount;
end

end

% Check that the buffer is not empty:
if byteBufferLength > 16
byteBufferStr = char(byteBuffer);

% Search for the magic number inside the buffer and check that at least one magic number has been found:
startIdx = strfind(byteBufferStr', char([2 1 4 3 6 5 8 7]));
if ~isempty(startIdx)

% Check the position of the first magic number and put it at
% the beginning of the buffer
if length(startIdx) >= 2
if startIdx(end-1) > 1
byteBuffer(1:byteBufferLength-(startIdx(1)-1)) = byteBuffer(startIdx(1):byteBufferLength);
byteBufferLength = byteBufferLength - (startIdx(1)-1);
end
else
if startIdx(1) > 1
byteBuffer(1:byteBufferLength-(startIdx(1)-1)) = byteBuffer(startIdx(1):byteBufferLength);
byteBufferLength = byteBufferLength - (startIdx(1)-1);
end
end
if byteBufferLength < 0
byteBufferLength = 0;
end

totalPacketLen = sum(byteBuffer(8+4+[1:4]) .* [1 256 65536 16777216]');
if ((byteBufferLength >= totalPacketLen) && (byteBufferLength ~= 0))
magicOk = 1;
else
magicOk = 0;
end
end
end

if (magicOk == 1)
%%%%% HEADER
word = [1 256 65536 16777216]';
idx = 0;
magicNumber = byteBuffer(idx + 1:8);
idx = idx + 8;
Header.version = dec2hex(sum(byteBuffer(idx+[1:4]) .* word));
idx = idx + 4;
Header.totalPacketLen = typecast(uint8(byteBuffer(idx+(1:4))),'uint32');
idx = idx + 4;
Header.platform = dec2hex(sum(byteBuffer(idx+[1:4]) .* word));
idx = idx + 4;
Header.frameNumber = typecast(uint8(byteBuffer(idx+(1:4))),'uint32');
frameNumber = Header.frameNumber;
idx = idx + 4;
Header.subFrameNumber = typecast(uint8(byteBuffer(idx+(1:4))),'uint32');
idx = idx + 4;
Header.chirpProcessingMargin = typecast(uint8(byteBuffer(idx+(1:4))),'uint32');
idx = idx + 4;
Header.frameProcessingMargin = typecast(uint8(byteBuffer(idx+(1:4))),'uint32');
idx = idx + 4;
Header.trackProcessTime = typecast(uint8(byteBuffer(idx+(1:4))),'uint32');
idx = idx + 4;
Header.uartSentTime = typecast(uint8(byteBuffer(idx+(1:4))),'uint32');
idx = idx + 4;
Header.numTLVs = typecast(uint8(byteBuffer(idx+(1:2))),'uint16');
idx = idx + 2;
Header.checksum = typecast(uint8(byteBuffer(idx+(1:2))),'uint16');
idx = idx + 2;


%%%%% TLV

% Analyze each of TLV messages:
for tlvIdx = 1:Header.numTLVs
% First, analyze the TLV header (TLV type and length):
tlv.type = typecast(uint8(byteBuffer(idx+(1:4))),'uint32');
idx = idx + 4;
tlv.length = typecast(uint8(byteBuffer(idx+(1:4))),'uint32');
idx = idx + 4;

% Check that the TLV message is of the right type (Detected objects):
switch tlv.type
case POINT_CLOUD_2D
numberOfPoints = (tlv.length - 28)/pointStruct_SIZE_BYTES;

pointUnit.elevationUnit = typecast(uint8(byteBuffer(idx+(1:4))),'single');
idx = idx + 4;
pointUnit.azimuthUnit = typecast(uint8(byteBuffer(idx+(1:4))),'single');
idx = idx + 4;
pointUnit.dopplerUnit = typecast(uint8(byteBuffer(idx+(1:4))),'single');
idx = idx + 4;
pointUnit.rangeUnit = typecast(uint8(byteBuffer(idx+(1:4))),'single');
idx = idx + 4;
pointUnit.snrUnit = typecast(uint8(byteBuffer(idx+(1:4))),'single');
idx = idx + 4;
pointStruct.elevation =[];
pointStruct.azimuth =[];
pointStruct.doppler =[];
pointStruct.range =[];
pointStruct.snr =[];

if numberOfPoints > 0
for i = 1:numberOfPoints
pointStruct.elevation = [pointStruct.elevation single(typecast(uint8(byteBuffer(idx+1)),'int8'))*pointUnit.elevationUnit];
idx = idx + 1;
pointStruct.azimuth = [pointStruct.azimuth single(typecast(uint8(byteBuffer(idx+1)),'int8'))*pointUnit.azimuthUnit];
idx = idx + 1;
pointStruct.doppler = [pointStruct.doppler single(typecast(uint8(byteBuffer(idx+(1:2))),'int16'))*pointUnit.dopplerUnit];
idx = idx + 2;
pointStruct.range = [pointStruct.range single(typecast(uint8(byteBuffer(idx+(1:2))),'uint16'))*pointUnit.rangeUnit];
idx = idx + 2;
pointStruct.snr = [pointStruct.snr single(typecast(uint8(byteBuffer(idx+(1:2))),'uint16'))*pointUnit.snrUnit];
idx = idx + 2;
end
dataOk = 1;
end
case TARGET_LIST_3D
numberOfTargets = (tlv.length - 28)/targetStruct2D_SIZE_BYTES;
if tlv.length > 0
targetStruct2D.tid = [];
targetStruct2D.posX = [];
targetStruct2D.posY = [];
targetStruct2D.velX = [];
targetStruct2D.velY = [];
targetStruct2D.accX = [];
targetStruct2D.accY = [];
targetStruct2D.posZ = [];
targetStruct2D.velZ = [];
targetStruct2D.accZ = [];

if numberOfPoints > 0
for i = 1:numberOfPoints
targetStruct2D.tid = [targetStruct2D.tid typecast(uint8(byteBuffer(idx+(1:4))),'uint32')];
idx = idx + 4;
targetStruct2D.posX = [targetStruct2D.posX typecast(uint8(byteBuffer(idx+(1:4))),'single')];
idx = idx + 4;
targetStruct2D.posY = [targetStruct2D.posY typecast(uint8(byteBuffer(idx+(1:4))),'single')];
idx = idx + 4;
targetStruct2D.velX = [targetStruct2D.velX typecast(uint8(byteBuffer(idx+(1:4))),'single')];
idx = idx + 4;
targetStruct2D.velY = [targetStruct2D.velY typecast(uint8(byteBuffer(idx+(1:4))),'single')];
idx = idx + 4;
targetStruct2D.accX = [targetStruct2D.accX typecast(uint8(byteBuffer(idx+(1:4))),'single')];
idx = idx + 4;
targetStruct2D.accY = [targetStruct2D.accY typecast(uint8(byteBuffer(idx+(1:4))),'single')];
idx = idx + 4;
targetStruct2D.posZ = [targetStruct2D.posZ typecast(uint8(byteBuffer(idx+(1:4))),'single')];
idx = idx + 4;
targetStruct2D.velZ = [targetStruct2D.velZ typecast(uint8(byteBuffer(idx+(1:4))),'single')];
idx = idx + 4;
targetStruct2D.accZ = [targetStruct2D.accZ typecast(uint8(byteBuffer(idx+(1:4))),'single')];
idx = idx + 4;
end
end

dataOk = 1;
end
case TARGET_INDEX
% rp = byteBuffer(idx+(1:tlv.length));
% idx = idx + tlv.length;
% rp=rp(1:2:end)+rp(2:2:end)*256;
end

end
%Remove processed data
if idx > 0
shiftSize = Header.totalPacketLen;
byteBuffer(1: byteBufferLength-shiftSize) = byteBuffer(shiftSize+1:byteBufferLength);
byteBufferLength = byteBufferLength - shiftSize;
if byteBufferLength < 0
% fprintf('Error: bytevec_cp_len < bytevecAccLen, %d %d \n', bytevec_cp_len, bytevecAccLen)
byteBufferLength = 0;
end
end
% if byteBufferLength > (byteBufferLength * 7/8)
% byteBufferLength = 0;
% end

else
magiNotOkCounter = magiNotOkCounter + 1;
end




示例程序

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
% Description: 
% Author: Yu Xuyao
% Email: yxy19991102@163.com
% Date: 2020-12-25 00:26:00
% LastEditTime: 2020-12-25 00:26:00
% LastEditors: Yu Xuyao

clear all
close all
clc

VisualReview = 1;
FallDetection = 0;

delete(instrfind);

configFile = "AOPconfig.cfg";
configcmd = readCfgFile(configFile);
[CLI_sphandle,DATA_sphandle] = setupUART('COM3','COM5');
sendCfg(CLI_sphandle,configcmd);

limit = 4;
N= 10000;

figure

H1 = uicontrol('Style', 'PushButton', ...
'String', 'Shutdown', ...
'Callback', 'shutVal = 1','InnerPosition',[7,30,60,20]);
H2 = uicontrol('Style', 'PushButton', ...
'String', 'Stop', ...
'Callback', 'stopVal = 1','InnerPosition',[7,60,60,20]);
H3 = uicontrol('Style', 'PushButton', ...
'String', 'Start', ...
'Callback', 'startVal = 1','InnerPosition',[7,90,60,20]);


h0 = scatter3([],[],[],'filled');
axis([-limit,limit,0,7,-1.5,1.5]);
xlabel('X (m)'); ylabel('Y (m)');zlabel('Z (m)');

myInd = 1;
dataOk = 0;
stopVal = 0;
startVal = 0;
shutVal = 0;
mask = 0;

frame = [];

while (myInd <= N)
dataOk = 0;

if mask ==1
pause(0.1);
else
[dataOk, frameNumber, pointStruct,targetStruct2D] = readAndParseData(DATA_sphandle);
end

if stopVal == 1
stopVal = 0;
mask = 1;
fprintf(CLI_sphandle, 'sensorStop\n');
end

if startVal == 1
startVal = 0;
mask = 0;
fprintf(CLI_sphandle, 'sensorStart 0\n');
end

if shutVal == 1
shutVal = 0;
fprintf(CLI_sphandle, 'sensorStop\n');
break;
end

if dataOk == 1
XData = [];
YData = [];
ZData = [];

graphXData = [];
graphYData = [];
graphZData = [];
% remove static points
processingSignal = [pointStruct.elevation ;pointStruct.azimuth;pointStruct.doppler;pointStruct.range;pointStruct.snr];

frame{myInd} = processingSignal';

if VisualReview
if myInd>3
showData = [frame{myInd}; frame{myInd-1}; frame{myInd-2}];
else
showData = frame{myInd};
end

temp = showData(:,4) .* cos(showData(:,1));
pos.XData = temp.*sin(showData(:,2));
pos.YData = temp.*cos(showData(:,2));
pos.ZData = showData(:,4) .* sin(showData(:,1));
pos.Doppler = showData(:,3);
pos.SNR = showData(:,5);

% Plotprocessed points
h0.XData = pos.XData;
h0.YData = pos.YData;
h0.ZData = pos.ZData;
h0.SizeData = 3*ones(1,length(pos.ZData));
h0.CData = abs(pos.Doppler);

title(['Frame No.' num2str(myInd)]);
drawnow limitrate;
end

pause(0.04);

myInd = myInd + 1;
end
end
delete(instrfind);
close all

前言

常规的Out Box of Demo的食用方法TI官方给的已经很多了总结下来就是:

  1. 电脑装emupack(设备和电脑通过USB通信的驱动程序)
  2. Uniflash给设备烧录demo的预编译文件
  3. 使用demo_visualizer与设备通过UART通信并读取数据, 可视化数据

但是这样使用官方软件的方法并不适合我们做更进一步的处理, 比如目标识别, 跟踪等等信息处理. 所以我们要做的就是写一个脚本来替代demo_visualizer. 那为了完成这样一件事儿我们需要知道demo_visualizer的工作机制, 具体通信的方式, 数据格式等等细节.

在这篇博客中会有许多概念我们默认为读者已经提前了解了, 比如通用异步收发传输器(Universal Asynchronous Receiver/Transmitter, UART), 基本的LFMCW雷达原理等. 如果有不清楚的概念搜索引擎搜索就可以得到需要的信息.

在这篇博客中我们将讲首先介绍out of box demo的数据传输方式以及格式, 之后会分别使用matlab和python给出一个例程, 这个例程可以读取雷达的数据还可以对数据进行可视化.

数据传输方式以及格式

数据传输方式

xWR设备使用UART与上位机通信,熟悉嵌入式的读者应该清楚从xWR SoC上的UART电平信号想通过USB传输一定需要一个转换电路将其转换为USB的差分信号, 同时还要能告诉USB host传输信号的源数据类型为UART. 那么这件工作是由谁来完成的呢, 对于xWR1642boost和xWR1843boost 是通过评估板上的一块XDS芯片完成的, 而对于xWR6843AoP, xWR6843ISK-ODS和xWR6843ISK 则是通过一块名为CP2015的UART bridge芯片完成的. 当你安装emupack这个驱动程序时, 其实你就安装了XDS的芯片驱动程序. 那CP2015的驱动程序呢? 由于CP2015是一块”烂大街”的芯片(使用极其广泛), 大多数Windows发行版在安装时就已经包含了这个驱动程序, 如果你的电脑不幸miss了CP2015的驱动程序, 通过网络你也可以很轻易的得到.

当你通过micro-USB的数据线将xWR评估板连接到你的电脑上时, 你会在设备管理器中看到检测到新插入的两个串行接口, 后面还会显示相应的COM标号:

  • xWR1642boost和xWR1843boost会显示为
  • xWR6843AoP, xWR6843ISK-ODS和xWR6843ISK会显示为

他们分别时CLI port和 Data port, CLI port传输你对demo程序的配置, 比如LFMCW雷达波形基础配置, 检测算法阈值的配置等等, 同时demo程序也会积极的与你交互, 告知你的配置格式数据是否合规. Data port传输经过雷达数据处理得到的检测点云数据, 芯片工作负载信息等等. 正是由于这两个串口不同的工作任务, 他们对信息传输的速度要求不同, CLI port工作在115200的波特率而Data port工作在926000的波特率. 当然这两个串口也是工作在全双工模式下的.

CLI port数据格式

CLI port是传输你对Demo程序的配置的端口, 配置信息的格式及相应的含义可以在

1
/ti/mmwave_sdk_<ver>/docs/mmwave_sdk_user_guide.pdf 

中的
1
3.4 Configuration (.cfg) File Format 

中找到

Data port数据格式

Data port的数据格式可以在

1
> \ti\mmwave_sdk_<ver>\packages\ti\demo\xwr68xx\mmw\docs

中找到,如下图所示

Data port数据输出格式是一个TLV格式的协议, 即就是每一个大的数据块(TLV)由 Tag, Length 和 Value 三个部分组成. 并且在 Data port 一次输出数据的过程中, 可能输出多个TLV, 这样在总的数据块头部, 需要一个总的Header数据段以告知该数据包TLV的数目以及这个Packet的长度(in bytes). 那么每个TLV传输的是什么数据呢? 比如检测的点云数据, 距离多普勒数据, 距离幅度的一维数据等等. 熟悉C语言的读者可以发现, 这样的多段线性的存储结构可以使用结构体来实现, 这也是TI 的官方Demo程序的做法. Packet 的header结构体叫 MmwDemo_output_message_header_t

1
2
3
4
5
6
7
8
9
10
11
12
13
14
typedef struct MmwDemo_output_message_header_t
{
uint16_t magicWord[4];
uint32_t version;
uint32_t totalPacketLen;
uint32_t platform;
uint32_t frameNumber;
uint32_t timeCpuCycles;
uint32_t numDetectedObj;
uint32_t numTLVs;
#ifdef SOC_XWR16XX
uint32_t subFrameNumber;
#endif
} MmwDemo_output_message_header;

每个结构体成员变量的含义可以在以下html文档中查看

1
/ti/mmwave_sdk_02_01_00_04/packages/ti/demo/xwr16xx/mmw/docs/doxygen/html/struct_mmw_demo__output__message__header__t.html

仅仅通过变量的名称我们可以发现, Packet的header包含了SDK的版本, Packet的长度, mmWave Device的名称, Packet的编号, 检测点的个数以及TLV的个数. 那么之后就是要确定每个TLV的格式了, Demo 中给出了几种功能的TLV

  1. List of detected objects
  2. Range profile
  3. Noise floor profile
  4. Azimuth static heatmap
  5. Range/Doppler heatmap
  6. Stats information

每种TLV的功能可以参见如下文档的 Output information sent to host 部分

1
/ti/mmwave_sdk_02_01_00_04/packages/ti/demo/xwr16xx/mmw/docs/doxygen/html/index.html

在这里着重要说明的是检测点云(List of detected objects)的输出格式, 也是我们需要重点需要从 Demo 中获得的数据

List of detected objects TLV 的数据结构如图

Tag和Length的结构体定义如下

1
2
3
4
5
typedef struct MmwDemo_output_message_tl_t
{
uint32_t type;
uint32_t length;
} MmwDemo_output_message_tl;

List of detected objects TLV的第二个结构体包含了检测到的点的个数与坐标存储的格式(Q-format, 即定点数)
1
2
3
4
5
typedef struct MmwDemo_output_message_dataObjDescr_t
{
uint16_t numDetetedObj;
uint16_t xyzQFormat;
} MmwDemo_output_message_dataObjDescr;

在MmwDemo_output_message_dataObjDescr之后紧跟着numDetetedObj个检测的点云数据
1
2
3
4
5
6
7
8
9
typedef volatile struct MmwDemo_detectedObj_t
{
uint16_t rangeIdx;
int16_t dopplerIdx;
uint16_t peakVal;
int16_t x;
int16_t y;
int16_t z;
} MmwDemo_detectedObj;

相关代码(matlab)

打开传输串口

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
function [CLI_sphandle,DATA_sphandle] = setupUART(CLIcomPortString,DATAcomPortString)

if strcmp(CLIcomPortString(1:3),'COM') || strcmp(DATAcomPortString(1:3),'COM')

CLI_sphandle = serial(CLIcomPortString,'BaudRate',115200);
set(CLI_sphandle,'Parity','none')
set(CLI_sphandle,'Terminator','LF')
fopen(CLI_sphandle);

DATA_sphandle = serial(DATAcomPortString,'BaudRate',921600);
set(DATA_sphandle,'Terminator', '');
set(DATA_sphandle,'InputBufferSize', 65536);
set(DATA_sphandle,'Timeout',10);
set(DATA_sphandle,'ErrorFcn',@dispError);
set(DATA_sphandle,'BytesAvailableFcnMode','byte');
set(DATA_sphandle,'BytesAvailableFcnCount', 2^16+1);%BYTES_AVAILABLE_FCN_CNT);
set(DATA_sphandle,'BytesAvailableFcn',@readUartCallbackFcn);
fopen(DATA_sphandle);
else
fprintf('COM port string format wrong\n');
end
end

CLI port配置数据

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
% Carrier frequency     GHz                          60.25
% Ramp Slope MHz/us 156
% Num ADC Samples 256
% ADC Sampling Rate Msps 12.5
% ADC Collection Time us 20.48
% Extra ramp time required (start time) us 3
% Chirp time (end time - start time) us 21
% Chirp duration (end time) us 24
% Sweep BW (useful) MHz 3194.88
% Total BW MHz 3744
% Max beat freq (80% of ADC sampling rate) MHz 10
% Max distance (80%) m 9.62
% Range resolution m 0.047
% Range resolution (meter per 1D-FFT bin) m/bin 0.047
%
% Inter-chirp duration us 7
% Number of chirp intervals in frame - 96
% Number of TX (TDM MIMO) 3
% Number of Tx elevation antennas 0
% Number of RX channels - 4
% Max umambiguous relative velocity kmph 48.19
% mileph 30.12
% Max extended relative velocity kmph 144.56
% mileph 90.35
% Frame time (total) ms 2.976
% Frame time (active) ms 2.304
% Range FFT size - 256
% Doppler FFT size - 32
% Radar data memory required KB 400
% Velocity resolution m/s 0.84
% Velocity resolution (m/s per 2D-FFT bin) m/s/bin 0.84
% Velocity Maximum m/s 13.39
% Extended Maximum Velocity m/s 40.16
% Maximum sweep accorss range bins range bin 0.85
%
sensorStop
flushCfg
dfeDataOutputMode 1
channelCfg 15 7 0
adcCfg 2 1
adcbufCfg -1 0 1 1 1
lowPower 0 0
profileCfg 0 60.25 7 3 24 0 0 156 1 256 12500 0 0 30
chirpCfg 0 0 0 0 0 0 0 1
chirpCfg 1 1 0 0 0 0 0 4
chirpCfg 2 2 0 0 0 0 0 2
frameCfg 0 2 32 0 100 1 0
guiMonitor -1 2 0 0 0 0 0
cfarCfg -1 0 2 8 4 3 0 11.0 0
cfarCfg -1 1 0 4 2 3 1 11.0 0
multiObjBeamForming -1 1 0.5
calibDcRangeSig -1 0 -5 8 256
clutterRemoval -1 1

compRangeBiasAndRxChanPhase 0.0 -1 0 1 0 1 0 -1 0 -1 0 1 0 1 0 -1 0 -1 0 1 0 1 0 -1 0
measureRangeBiasAndRxChanPhase 0 1. 0.2

aoaFovCfg -1 -90 90 -90 90
cfarFovCfg -1 0 0.1 5
cfarFovCfg -1 1 -13.39 13.39
extendedMaxVelocity -1 0

CQRxSatMonitor 0 3 4 63 0
CQSigImgMonitor 0 127 4
analogMonitor 0 0
lvdsStreamCfg -1 0 0 0
bpmCfg -1 0 0 0
sensorStart

读取CLI port配置数据

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
function [configcmd] = readCfgFile(configFile)

configcmd = cell(1,100);
fid = fopen(configFile, 'r');
if fid == -1
fprintf('File %s not found!\n', configFile);
return;
else
fprintf('Opening configuration file %s ...\n', configFile);
end
tline = fgetl(fid);
k=1;
while ischar(tline)
if ~isempty(tline)
configcmd{k} = tline;
k = k + 1;
end
tline = fgetl(fid);
end
configcmd = configcmd(1:k-1);
fclose(fid);
end

发送CLI port的配置数据

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
function sendCfg(UART_sphandle,configcmd)
mmwDemoCliPrompt = char('mmwDemo:/>');

%Send CLI configuration to IWR
fprintf('Sending configuration to IWR radar\n');
for k=1:length(configcmd)
command = configcmd{k};
fprintf(UART_sphandle, command);
fprintf('%s\n', command);
echo = fgetl(UART_sphandle); % Get an echo of a command
done = fgetl(UART_sphandle); % Get "Done"
fprintf('%s\n', done);
prompt = fread(UART_sphandle, size(mmwDemoCliPrompt,2)); % Get the prompt back
fprintf('%s\n', prompt);
end
end

读取Data port数据

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
function [dataOk, frameNumber, detObj] = readAndParseData(DATA_sphandle)
OBJ_STRUCT_SIZE_BYTES = 16;
BYTE_VEC_ACC_MAX_SIZE = 2^16;
MMWDEMO_UART_MSG_DETECTED_POINTS = 1;
MMWDEMO_UART_MSG_RANGE_PROFILE = 2;
MMWDEMO_UART_MSG_DETECTED_POINTS_SIDE_INFO = 7;
maxBufferSize = BYTE_VEC_ACC_MAX_SIZE;

detObj = [];
frameNumber = 0;

persistent byteBuffer
if isempty(byteBuffer)
byteBuffer = zeros(maxBufferSize,1);
end

persistent byteBufferLength
if isempty(byteBufferLength)
byteBufferLength = 0;
end

persistent magiNotOkCounter
if isempty(magiNotOkCounter)
magiNotOkCounter = 0;
end

magicOk = 0;
dataOk = 0;

bytesToRead = get(DATA_sphandle,'BytesAvailable');
if (bytesToRead ~= 0)
% Read the Data Serial Port
[bytevec, byteCount] = fread(DATA_sphandle, bytesToRead, 'uint8');

% Check if the buffer is not full, and then add the data to the buffer:
if(byteBufferLength + byteCount < maxBufferSize)
byteBuffer(byteBufferLength+1:byteBufferLength + byteCount) = bytevec(1:byteCount);
byteBufferLength = byteBufferLength + byteCount;
end

end

% Check that the buffer is not empty:
if byteBufferLength > 16
byteBufferStr = char(byteBuffer);

% Search for the magic number inside the buffer and check that at least one magic number has been found:
startIdx = strfind(byteBufferStr', char([2 1 4 3 6 5 8 7]));
if ~isempty(startIdx)

% Check the position of the first magic number and put it at
% the beginning of the buffer
if length(startIdx) >= 2
if startIdx(end-1) > 1
byteBuffer(1:byteBufferLength-(startIdx(1)-1)) = byteBuffer(startIdx(1):byteBufferLength);
byteBufferLength = byteBufferLength - (startIdx(1)-1);
end
else
if startIdx(1) > 1
byteBuffer(1:byteBufferLength-(startIdx(1)-1)) = byteBuffer(startIdx(1):byteBufferLength);
byteBufferLength = byteBufferLength - (startIdx(1)-1);
end
end
if byteBufferLength < 0
byteBufferLength = 0;
end

totalPacketLen = sum(byteBuffer(8+4+[1:4]) .* [1 256 65536 16777216]');
if ((byteBufferLength >= totalPacketLen) && (byteBufferLength ~= 0))
magicOk = 1;
else
magicOk = 0;
end
end
end

if (magicOk == 1)
%%%%% HEADER
word = [1 256 65536 16777216]';
idx = 0;
magicNumber = byteBuffer(idx + 1:8);
idx = idx + 8;
Header.version = dec2hex(sum(byteBuffer(idx+[1:4]) .* word));
idx = idx + 4;
Header.totalPacketLen = typecast(uint8(byteBuffer(idx+[1:4])),'uint32');
idx = idx + 4;
Header.platform = dec2hex(sum(byteBuffer(idx+[1:4]) .* word));
idx = idx + 4;
Header.frameNumber = typecast(uint8(byteBuffer(idx+[1:4])),'uint32');
frameNumber = Header.frameNumber;
idx = idx + 4;
Header.timeCpuCycles = typecast(uint8(byteBuffer(idx+[1:4])),'uint32');
idx = idx + 4;
Header.numDetectedObj = typecast(uint8(byteBuffer(idx+[1:4])),'uint32');
idx = idx + 4;
Header.numTLVs = typecast(uint8(byteBuffer(idx+[1:4])),'uint32');
idx = idx + 4;
Header.subFrameNumber = typecast(uint8(byteBuffer(idx+[1:4])),'uint32');
idx = idx + 4;


%%%%% TLV

% Analyze each of TLV messages:
for tlvIdx = 1:Header.numTLVs
word = [1 256 65536 16777216]';
% First, analyze the TLV header (TLV type and length):
tlv.type = sum(byteBuffer(idx+uint32(1:4)) .* word);
idx = idx + 4;
tlv.length = sum(byteBuffer(idx+uint32(1:4)) .* word);
idx = idx + 4;

% Check that the TLV message is of the right type (Detected objects):
switch tlv.type
case MMWDEMO_UART_MSG_DETECTED_POINTS
detObj =[];

if tlv.length > 0
% Extract the raw data for all the detected points
bytes = byteBuffer(idx+(1:Header.numDetectedObj*OBJ_STRUCT_SIZE_BYTES));
idx = idx + Header.numDetectedObj*OBJ_STRUCT_SIZE_BYTES;

% Reshape the array to have the data for each point
% (X,Y,Z,doppler) in each column
bytes = reshape(bytes, OBJ_STRUCT_SIZE_BYTES, Header.numDetectedObj);

% Convert the byte matrix to float data
floatData = reshape(typecast(reshape(uint8(bytes), 1, []), 'single'),4,Header.numDetectedObj);

detObj.numObj = Header.numDetectedObj;
detObj.x = floatData(1,:);
detObj.y = floatData(2,:);
detObj.z = floatData(3,:);
detObj.doppler = floatData(4,:);

dataOk = 1;

end
case MMWDEMO_UART_MSG_DETECTED_POINTS_SIDE_INFO

if tlv.length > 0
bytes = byteBuffer(idx+(1:Header.numDetectedObj*4));
idx = idx + Header.numDetectedObj*4;

% Reshape the array to have the data for each point
% (snr,noise) in each column
bytes = reshape(bytes, 4, Header.numDetectedObj);

% Convert the byte matrix to float data
floatData = reshape(typecast(reshape(uint8(bytes), 1, []), 'int16'),2,Header.numDetectedObj);
detObj.snr = floatData(1,:);
detObj.noise = floatData(2,:);

dataOk = 1;

end
case MMWDEMO_UART_MSG_RANGE_PROFILE
rp = byteBuffer(idx+uint32(1:tlv.length));
idx = idx + tlv.length;
rp=rp(1:2:end)+rp(2:2:end)*256;
end

end
%Remove processed data
if idx > 0
shiftSize = Header.totalPacketLen;
byteBuffer(1: byteBufferLength-shiftSize) = byteBuffer(shiftSize+1:byteBufferLength);
byteBufferLength = byteBufferLength - shiftSize;
if byteBufferLength < 0
byteBufferLength = 0;
end
end

else
magiNotOkCounter = magiNotOkCounter + 1;
end

示例程序

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
clear all; close all
delete(instrfind);

N = 10000;
limit = 5;

% Setup radar with the parameters from the configuration file
configFile = "profile_3d.cfg";
configcmd = readCfgFile(configFile);
[CLI_sphandle,DATA_sphandle] = setupUART('COM9','COM10');
sendCfg(CLI_sphandle,configcmd)

%% Initialize the figure
figure

H1 = uicontrol('Style', 'PushButton', ...
'String', 'Shutdown', ...
'Callback', 'shutVal = 1','InnerPosition',[7,30,60,20]);
H2 = uicontrol('Style', 'PushButton', ...
'String', 'Stop', ...
'Callback', 'stopVal = 1','InnerPosition',[7,60,60,20]);
H3 = uicontrol('Style', 'PushButton', ...
'String', 'Start', ...
'Callback', 'startVal = 1','InnerPosition',[7,90,60,20]);
h = scatter3([],[],[],'filled');
axis([-limit,limit,0,limit,-limit,limit]);
xlabel('X (m)'); ylabel('Y (m)');

%% main loop

myInd = 1;
lastreadTime = 0;
dataOk = 0;
stopVal = 0;
startVal = 0;
shutVal = 0;
mask = 0;

tic
while (myInd <= N)
dataOk = 0;

% Read the data from the radar:
if mask ==1
pause(0.1);
else
[dataOk, frameNumber, detObj] = readAndParseData(DATA_sphandle);
end

if stopVal == 1
stopVal = 0;
mask = 1;
fprintf(CLI_sphandle, 'sensorStop\n');
end

if startVal == 1
startVal = 0;
mask = 0;
fprintf(CLI_sphandle, 'sensorStart 0\n');
end

if shutVal == 1
shutVal = 0;
fprintf(CLI_sphandle, 'sensorStop\n');
break;
end

if dataOk == 1

% Store all the data from the radar
frame{myInd} = detObj;

%% do process here

%% above

% Plot the radar points
h.XData = detObj.x;
h.YData = detObj.y;
h.ZData = detObj.z;
drawnow limitrate;
pause(0.05);

myInd = myInd + 1;
end
end

delete(instrfind);
close all

相关代码(Python)

依赖

在Python的代码中需要安装如下几个依赖:

  1. pyserial: 用于通过串口与设备进行通信
  2. numpy: 进行矩阵的操作和部分数学处理
  3. Pyqt, Pyqtgraph, OpenGL: 对数据进行可视化操作并显示
  4. sklearn: 聚类等操作
  5. pandas: 数据管理

    代码

    在这个代码中其实进行了DBscan聚类的处理, 感兴趣的读者可以多多尝试其他的信息处理方法
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90
    91
    92
    93
    94
    95
    96
    97
    98
    99
    100
    101
    102
    103
    104
    105
    106
    107
    108
    109
    110
    111
    112
    113
    114
    115
    116
    117
    118
    119
    120
    121
    122
    123
    124
    125
    126
    127
    128
    129
    130
    131
    132
    133
    134
    135
    136
    137
    138
    139
    140
    141
    142
    143
    144
    145
    146
    147
    148
    149
    150
    151
    152
    153
    154
    155
    156
    157
    158
    159
    160
    161
    162
    163
    164
    165
    166
    167
    168
    169
    170
    171
    172
    173
    174
    175
    176
    177
    178
    179
    180
    181
    182
    183
    184
    185
    186
    187
    188
    189
    190
    191
    192
    193
    194
    195
    196
    197
    198
    199
    200
    201
    202
    203
    204
    205
    206
    207
    208
    209
    210
    211
    212
    213
    214
    215
    216
    217
    218
    219
    220
    221
    222
    223
    224
    225
    226
    227
    228
    229
    230
    231
    232
    233
    234
    235
    236
    237
    238
    239
    240
    241
    242
    243
    244
    245
    246
    247
    248
    249
    250
    251
    252
    253
    254
    255
    256
    257
    258
    259
    260
    261
    262
    263
    264
    265
    266
    267
    268
    269
    270
    271
    272
    273
    274
    275
    276
    277
    278
    279
    280
    281
    282
    283
    284
    285
    286
    287
    288
    289
    290
    291
    292
    293
    294
    295
    296
    297
    298
    299
    300
    301
    302
    303
    304
    305
    306
    307
    308
    309
    310
    311
    312
    313
    314
    315
    316
    317
    318
    319
    320
    321
    322
    323
    324
    325
    326
    327
    328
    329
    330
    331
    332
    333
    334
    335
    336
    337
    338
    339
    340
    341
    342
    343
    344
    345
    346
    347
    348
    349
    350
    351
    352
    353
    354
    355
    356
    357
    358
    359
    360
    361
    362
    363
    364
    #
    # File: 读取IWR6843处理得到的点云数据(3维)进行目标跟踪,同时调用openGL对点云进行显示
    # Author: UESTC Yu Xuyao
    # Email: yxy19991102@163.com
    #

    import serial
    import time
    import numpy as np
    import pyqtgraph as pg
    from pyqtgraph.Qt import QtGui
    import pyqtgraph.opengl as gl
    # from IWR.MOT3D import *
    import numpy
    from sklearn.cluster import dbscan
    import pandas

    # Change the configuration file named
    configFileName = 'profile_iwr6843_ods_3d.cfg'
    # configFileName = 'profile_3d.cfg'

    CLIport = {}
    Dataport = {}
    byteBuffer = np.zeros(2 ** 15, dtype='uint8')
    byteBufferLength = 0

    testroot = []
    confirmedroot = []
    Z_k_prev = numpy.mat([])

    cnt = 1
    show = []
    changdu = 5


    # ------------------------------------------------------------------
    # Function to configure the serial ports and send the data from
    # the configuration file to the radar
    def serialConfig(configFileName):
    global CLIport
    global Dataport
    # Open the serial ports for the configuration and the data ports

    # Raspberry pi
    # CLIport = serial.Serial('/dev/ttyACM0', 115200)
    # Dataport = serial.Serial('/dev/ttyACM1', 921600)

    # Windows
    CLIport = serial.Serial('COM3', 115200)
    Dataport = serial.Serial('COM4', 921600)
    # CLIport = serial.Serial('COM22', 115200)
    # Dataport = serial.Serial('COM21', 921600)

    # Read the configuration file and send it to the board
    config = [line.rstrip('\r\n') for line in open(configFileName)]
    for i in config:
    print(i)
    if len(i)>0:
    if i[0] != '%':
    CLIport.write((i + '\n').encode())
    time.sleep(0.1)
    reply = CLIport.read(CLIport.in_waiting).decode()
    print(reply)

    return CLIport, Dataport


    # ------------------------------------------------------------------

    # Function to parse the data inside2 the configuration file
    def parseConfigFile(configFileName):
    configParameters = {} # Initialize an empty dictionary to store the configuration parameters

    # Read the configuration file and send it to the board
    config = [line.rstrip('\r\n') for line in open(configFileName)]
    for i in config:

    # Split the line
    splitWords = i.split(" ")

    # Hard code the number of antennas, change if other configuration is used
    numRxAnt = 4
    numTxAnt = 3

    # Get the information about the profile configuration
    if "profileCfg" in splitWords[0]:
    startFreq = int(float(splitWords[2]))
    idleTime = int(splitWords[3])
    rampEndTime = float(splitWords[5])
    freqSlopeConst = float(splitWords[8])
    numAdcSamples = int(splitWords[10])
    numAdcSamplesRoundTo2 = 1

    while numAdcSamples > numAdcSamplesRoundTo2:
    numAdcSamplesRoundTo2 = numAdcSamplesRoundTo2 * 2

    digOutSampleRate = int(splitWords[11])

    # Get the information about the frame configuration
    elif "frameCfg" in splitWords[0]:
    chirpStartIdx = int(splitWords[1])
    chirpEndIdx = int(splitWords[2])
    numLoops = int(splitWords[3])
    numFrames = int(splitWords[4])
    framePeriodicity = float(splitWords[5])

    # Combine the read data to obtain the configuration parameters
    numChirpsPerFrame = (chirpEndIdx - chirpStartIdx + 1) * numLoops
    configParameters["numDopplerBins"] = numChirpsPerFrame / numTxAnt
    configParameters["numRangeBins"] = numAdcSamplesRoundTo2
    configParameters["rangeResolutionMeters"] = (3e8 * digOutSampleRate * 1e3) / (
    2 * freqSlopeConst * 1e12 * numAdcSamples)
    configParameters["rangeIdxToMeters"] = (3e8 * digOutSampleRate * 1e3) / (
    2 * freqSlopeConst * 1e12 * configParameters["numRangeBins"])
    configParameters["dopplerResolutionMps"] = 3e8 / (
    2 * startFreq * 1e9 * (idleTime + rampEndTime) * 1e-6 * configParameters["numDopplerBins"] * numTxAnt)
    configParameters["maxRange"] = (300 * 0.9 * digOutSampleRate) / (2 * freqSlopeConst * 1e3)
    configParameters["maxVelocity"] = 3e8 / (4 * startFreq * 1e9 * (idleTime + rampEndTime) * 1e-6 * numTxAnt)

    return configParameters


    # ------------------------------------------------------------------

    # Funtion to read and parse the incoming data
    def readAndParseData68xx(Dataport, configParameters):
    global byteBuffer, byteBufferLength

    # Constants
    OBJ_STRUCT_SIZE_BYTES = 12
    BYTE_VEC_ACC_MAX_SIZE = 2 ** 15
    MMWDEMO_UART_MSG_DETECTED_POINTS = 1
    MMWDEMO_UART_MSG_RANGE_PROFILE = 2
    maxBufferSize = 2 ** 15
    tlvHeaderLengthInBytes = 8
    pointLengthInBytes = 16
    magicWord = [2, 1, 4, 3, 6, 5, 8, 7]

    # Initialize variables
    magicOK = 0 # Checks if magic number has been read
    dataOK = 0 # Checks if the data has been read correctly
    frameNumber = 0
    detObj = {}

    readBuffer = Dataport.read(Dataport.in_waiting)
    byteVec = np.frombuffer(readBuffer, dtype='uint8')
    byteCount = len(byteVec)

    # Check that the buffer is not full, and then add the data to the buffer
    if (byteBufferLength + byteCount) < maxBufferSize:
    byteBuffer[byteBufferLength:(byteBufferLength + byteCount)] = byteVec[0:byteCount]
    byteBufferLength = byteBufferLength + byteCount

    # Check that the buffer has some data
    if byteBufferLength > 16:

    # Check for all possible locations of the magic word
    possibleLocs = np.where(byteBuffer == magicWord[0])[0]

    # Confirm that is the beginning of the magic word and store the index in startIdx
    startIdx = []
    for loc in possibleLocs:
    check = byteBuffer[loc:loc + 8]
    if np.all(check == magicWord):
    startIdx.append(loc)

    # Check that startIdx is not empty
    if startIdx:

    # Remove the data before the first start index
    if 0 < startIdx[0] < byteBufferLength:
    byteBuffer[:byteBufferLength - startIdx[0]] = byteBuffer[startIdx[0]:byteBufferLength]
    byteBuffer[byteBufferLength - startIdx[0]:] = np.zeros(len(byteBuffer[byteBufferLength - startIdx[0]:]),
    dtype='uint8')
    byteBufferLength = byteBufferLength - startIdx[0]

    # Check that there have no errors with the byte buffer length
    if byteBufferLength < 0:
    byteBufferLength = 0

    # Read the total packet length
    totalPacketLen = int.from_bytes(byteBuffer[12:12 + 4], byteorder='little')

    # Check that all the packet has been read
    if (byteBufferLength >= totalPacketLen) and (byteBufferLength != 0):
    magicOK = 1

    # If magicOK is equal to 1 then process the message
    if magicOK:

    # Initialize the pointer index
    idX = 0

    # Read the header
    magicNumber = byteBuffer[idX:idX + 8]
    idX += 8
    version = format(int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little'), 'x')
    idX += 4
    totalPacketLen = int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little')
    idX += 4
    platform = format(int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little'), 'x')
    idX += 4
    frameNumber = int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little')
    idX += 4
    timeCpuCycles = int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little')
    idX += 4
    numDetectedObj = int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little')
    idX += 4
    numTLVs = int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little')
    idX += 4
    subFrameNumber = int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little')
    idX += 4

    # Read the TLV messages
    for tlvIdx in range(numTLVs):

    # Check the header of the TLV message
    tlv_type = int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little')
    idX += 4
    tlv_length = int.from_bytes(byteBuffer[idX:idX + 4], byteorder='little')
    idX += 4

    # Read the data depending on the TLV message
    if tlv_type == MMWDEMO_UART_MSG_DETECTED_POINTS:

    # Initialize the arrays
    x = np.zeros(numDetectedObj, dtype=np.float32)
    y = np.zeros(numDetectedObj, dtype=np.float32)
    z = np.zeros(numDetectedObj, dtype=np.float32)
    velocity = np.zeros(numDetectedObj, dtype=np.float32)

    for objectNum in range(numDetectedObj):
    # Read the data for each object
    x[objectNum] = byteBuffer[idX:idX + 4].view(dtype=np.float32)
    idX += 4
    y[objectNum] = byteBuffer[idX:idX + 4].view(dtype=np.float32)
    idX += 4
    z[objectNum] = byteBuffer[idX:idX + 4].view(dtype=np.float32)
    idX += 4
    velocity[objectNum] = byteBuffer[idX:idX + 4].view(dtype=np.float32)
    idX += 4

    # Store the data in the detObj dictionary
    detObj = {"numObj": numDetectedObj, "x": x, "y": y, "z": z, "velocity": velocity}
    dataOK = 1

    # Remove already processed data
    if 0 < idX < byteBufferLength:
    shiftSize = totalPacketLen

    byteBuffer[:byteBufferLength - shiftSize] = byteBuffer[shiftSize:byteBufferLength]
    byteBuffer[byteBufferLength - shiftSize:] = np.zeros(len(byteBuffer[byteBufferLength - shiftSize:]),
    dtype='uint8')
    byteBufferLength = byteBufferLength - shiftSize

    # Check that there are no errors with the buffer length
    if byteBufferLength < 0:
    byteBufferLength = 0

    return dataOK, frameNumber, detObj


    # ------------------------------------------------------------------

    # Funtion to update the data and display in the plot
    def update():
    dataOk = 0
    global detObj, confirmedroot, testroot, Z_k_prev, GLScatterPlot
    global show, cnt
    x = []
    y = []
    z = []
    mask = 1

    # Read and parse the received data
    dataOk, frameNumber, detObj = readAndParseData68xx(Dataport, configParameters)

    if dataOk:
    x = detObj["x"]
    y = detObj["y"]
    z = detObj["z"]

    if len(x) != 0:
    try:
    core_samples, cluster_ids = dbscan(np.vstack((x, y, z)).T, eps=0.3, min_samples=5)
    except:
    # print(np.vstack((x, y, z)).T)
    return dataOk
    df = pandas.DataFrame(np.c_[np.vstack((x, y, z)).T, cluster_ids],
    columns=['x', 'y', 'z', 'cluster_id'])
    df = df[df.cluster_id != -1]
    count = df['cluster_id'].value_counts()

    x = []
    y = []
    z = []
    if count.shape[0] >= 1:
    for i in range(0, count.shape[0]):
    df1 = df[df.cluster_id == i]
    # mean = np.mean([df1['x'], df1['y'], df1['z']], 1)
    x = numpy.hstack((x, df1['x'].values))
    y = numpy.hstack((y, df1['y'].values))
    z = numpy.hstack((z, df1['z'].values))
    # Z_k = np.vstack((x, y, z)).T
    # if cnt==1:
    # show = Z_k
    # cnt += 1
    # elif cnt<changdu:
    # show = numpy.vstack((show,Z_k))
    # cnt+=1
    # elif cnt==changdu:
    # cnt = 1
    # show = numpy.vstack((show, Z_k))
    # GLScatterPlot.setData(pos=show)

    Z_k = np.vstack((x, y, z)).T
    GLScatterPlot.setData(pos=Z_k, color=(1., 1., 1., .9))

    # confirmedroot, testroot, Z_k_prev = MOT(Z_k, Z_k_prev, confirmedroot, testroot)

    # if len(confirmedroot) > 0:
    # tmppos = confirmedroot[0].est[0:3, :].T.A
    # for i in range(1,len(confirmedroot)):
    # tmppos = numpy.vstack((tmppos,confirmedroot[i].est[0:3, :].T.A))
    # LinePlot.setData(pos=tmppos)
    # QtGui.QApplication.processEvents()
    # else:
    # LinePlot.setData(pos=np.array([100, 100, 100]))
    # QtGui.QApplication.processEvents()


    # ------------------------- MAIN -----------------------------------------

    # Configurate the serial port
    CLIport, Dataport = serialConfig(configFileName)

    # Get the configuration parameters from the configuration file
    configParameters = parseConfigFile(configFileName)

    # START QtAPPfor the plot
    app = QtGui.QApplication([])

    app = pg.mkQApp()

    view = gl.GLViewWidget()

    view.show()

    axis = gl.GLAxisItem()
    axis.scale(10, 10, 10)

    view.addItem(axis)

    GLScatterPlot = gl.GLScatterPlotItem()
    view.addItem(GLScatterPlot)

    # Main loop

    timer = pg.QtCore.QTimer()
    timer.timeout.connect(update) # 定时调用plotData函数
    timer.start(50) # 多少ms调用一次

    app.exec_()

最近一直在follow Yingying ChenDina Katabi组的工作,那就先从19年Yingying Chen组里Jian Liu等人发的survey开始看起

Wireless Sensing for Human Activity: A Survey(10.1109/COMST.2019.2934489)

目前典型的Wireless Sensing的workflow

对WIFI sensing的SOA大致分成了4类

  1. intrusion detection & occupancy monitoring
  2. activ-ity & gesture recognition
  3. vital signs monitoring
  4. useridentification & localization.

对Wireless sensing中的关键技术也是4类

  1. Received signal strength indicator (RSSI)
  2. channel stateinformation (CSI)
  3. frequency shift for frequency modulatedcarrier wave (FMCW)
  4. Doppler shift

Defination

desired output $\boldsymbol{d}\in \mathbb{R}^{1\times 1}$

1st laywer input $\boldsymbol{x}^{(1)}\in \mathbb{R}^{n\times 1}$

1st laywer weight $\boldsymbol{W}^{(1)}\in \mathbb{R}^{m\times n}$

1st laywer bias $\boldsymbol{b}^{(1)}\in \mathbb{R}^{m\times 1}$

1st laywer output $\boldsymbol{z}^{(1)}\in \mathbb{R}^{m\times 1}$

2st laywer input $\boldsymbol{x}^{(2)}\in \mathbb{R}^{m\times 1}$

2nd laywer weight $\boldsymbol{W}^{(2)}\in \mathbb{R}^{1\times m}$

2nd laywer bias $\boldsymbol{b}^{(2)}\in \mathbb{R}^{1\times 1}$

2st laywer output $\boldsymbol{z}^{(2)}\in \mathbb{R}^{1\times 1}$

output $\boldsymbol{y} \in \mathbb{R}^{1\times 1}$

Activation function:

  1. sigmod

  2. RELU

forward propagation

Define Loss function$J(\boldsymbol{\cdot})$

Take the derivative of the loss function

Back propagation

Learning step $\mu$

Simulation

JPDA的基本模型

确认矩阵

比如有这样一个情况
fa2fa2ce95be726b453492865a474ed6.png
那么根据规则可以建立确认矩阵

互联矩阵(联合事件)

联合事件与互联矩阵一一对应
所谓互联矩阵,用之前的例子举例就是将$\Omega$拆分成

互联概率的计算

符号定义:

  1. $\theta_i(k)$:表示第 $i$ 个联合事件

  2. $\hat{\Omega}\left[\theta_{i}(k)\right]$:表示与第 $i$ 个联合事件对应的互联矩阵

    根据上述两个基本假设容易推出互联矩阵满足:

    用文字叙述就是:
    每一个量测只来源于一个目标或者是杂波虚警
    每一个目标只有一个或者没有量测

  3. $\theta_{jt}(k)$:测量 $j$ 源于目标 $t(1\leq t \leq T)$ 的事件,$\theta_{0t}$ 表示目标是杂波或者虚警

  4. $\beta_{jt}(k)$:表示第 $j$ 个测量与 $t$ 互联的概率,且

  5. $\theta_{j t}^{i}(k)$:量测 $j$ 在第 $i$ 个联合事件中源于目标 $t(0 \leqslant t \leqslant T)$ 的事件

  6. $\hat{\omega}_{j t}^{i}\left(\theta_{i}(k)\right)$:表示在第 i 个联合事件中,量测 j 是否源于目标 $t,$ 在量测 j 源于目标 $t$ 时为 $1,$ 否则为 0

状态估计计算
则 $k$ 时刻目标 $t$ 的状态估计为

式中

表示在 $k$ 时刻用第 $j$ 个量测对目标 $t$ 进行卡尔曼滤波所得的状态估计。
$\hat{\boldsymbol{X}}_{0}^{t}(k \mid k)$ 表示 $k$时刻没有量测源于目标的情况,这时需要用预测值 $\hat{\boldsymbol{X}}^{t}(k \mid k-1)$ 来代替,即:

概率计算
第 $j$ 个量测与目标互联的概率可利用下式求取:

可以看到求取状态估计 $\hat{\boldsymbol{X}}^{t}(k \mid k)$ 的关键就是计算联合事件概率(后验条件概率) $\operatorname{Pr}\left\{\theta_{i}(k) \mid Z^{k}\right\}$

联合事件概率计算

为了以后讨论问题的方便,这里引入两个二元指示变量:
(1) 量测互联指示,即

表示量测 j 在联合事件 $\theta_{i}(k)$ 中是否跟一个真实目标互联:
(2) 目标检测指示,即

表示任一量测在联合事件 $\theta_{i}(k)$ 中是否与目标 $t$ 互联,也即目标 $t$ 是否被检测。
设 $\phi\left[\theta_{i}(k)\right]$ 表示在联合事件 $\theta_{i}(k)$ 中假量测的数量,则

核心问题:求解 $\operatorname{Pr}\left\{\theta_{i}(k) \mid Z^{k}\right\}$

一看这形式老朋友了,贝叶斯推论三步走起

式中,c 为归一化常数

似然函数:

其中

联合事件 $\theta_{i}(k)$ 的先验概率:

其中

最后得到

状态估计协方差计算

没有测量来自目标

有测量来自目标

算法仿真

在航迹起始的基础上进行仿真

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
clc
clear all
close all

load('Root.mat');
load('points.mat');

points = points(11:end);
n = size(points,1);

n_z = 2;
gamma = chi2inv(0.99,n_z);
P_G = PG(gamma,n_z);

T = 1;
F = [1 0 T 0;
0 1 0 T;
0 0 1 0;
0 0 0 1];
H = [1 0 0 0;
0 1 0 0];
Gamma=[T*T/2 0;0 T*T/2;T 0;0 T];
Q = 0.01*Gamma*Gamma';%根据实际过程噪声的协方差更改
R = 5*H*H'; %根据实际测量误差的协方差更改

for kk =1:n
Z_k = cell2mat(points(kk));
t = size(Root,2); % 目标数
mk = size(Z_k,2); %有效测量数

for i = 1:t
P(:,:,i)=F*Root(i).pkk*F'+Q;
S(:,:,i) = H*P(:,:,i)*H' + R;
K(:,:,i)= P(:,:,i)*H'*inv(S(:,:,i));
Z_predict(:,i) = H*Root(i).x_predict;
Volume(i)=ellipse_Volume(gamma,n_z,S(:,:,i));
end

%% 建立确认矩阵
Omega = zeros(mk,t+1);%确认矩阵
Z_set = [];
for j = 1:t
for k = 1:mk
V=(Z_k(:,k)-Z_predict(:,j))'*inv(S(:,:,j))*(Z_k(:,k)-Z_predict(:,j));
if (V<=gamma)
Omega(k,1) = 1;
Omega(k,j+1) = 1;
Z_set = [Z_set k]; % 记录确认矩阵对应的量测 索引
end
end
end
Z_set = unique(Z_set,'stable');
Omega(all(Omega==0,2),:)=[]; %删除确认矩阵的零行

%% 建立互联矩阵
hat_omega=zeros(size(Omega,1),size(Omega,2),10000);
hat_omega(:,1,1:10000)=1;
num=1;
for i=1:size(Omega,1)
if Omega(i,2)==1
hat_omega(i,2,num)=1;hat_omega(i,1,num)=0;
num=num+1;
for j=1:size(Omega,1)
if (i~=j)&&(Omega(j,3)==1)
hat_omega(i,2,num)=1;hat_omega(i,1,num)=0;
hat_omega(j,3,num)=1;hat_omega(j,1,num)=0;
num=num+1;
for k=1:size(Omega,1)
if (k~=i)&&(k~=j)&&(Omega(k,4)==1)
hat_omega(i,2,num)=1;hat_omega(i,1,num)=0;
hat_omega(j,3,num)=1;hat_omega(j,1,num)=0;
hat_omega(k,4,num)=1;hat_omega(k,1,num)=0;
num=num+1;
end
end
end
end
end
end
hat_omega=hat_omega(:,:,1:num);

%% 计算theta_i 条件概率
P_D = 0.5;
Pr = PrthetaZk(hat_omega,t,Z_k,Z_set,Z_predict,S,P_D,sum(Volume));

%% 计算互联概率 beta
beta = zeros(length(Z_set)+1,t);
for i=1:t
for j=1:length(Z_set)
for k=1:num
beta(j+1,i)=beta(j+1,i)+Pr(k)*hat_omega(j,i+1,k);
end
end
end
% 计算j = 0的特殊情况,同时满足归一化
beta(1,:) = 1-sum(beta(2:end,:));

%% 计算状态估计X_jt
X_jt = cell(length(Z_set)+1,t);

% 计算j = 0的特殊情况
for i = 1:t
X_jt(1,i) = {Root(i).x_predict};
end

for i = 1:t
for j = 2:length(Z_set)+1
X_jt(j,i) = {Root(i).x_predict + K(:,:,i)*(Z_k(:,Z_set(j-1)) - Z_predict(:,i))};
end
end

X_t = zeros(size(Root(1).x_predict,1),t);
temp = zeros([size(Root(1).pkk) t]);
for i = 1:t
%% 计算状态估计X_t
for j = 1:length(Z_set)+1
X_t(:,i) = X_t(:,i) + beta(j,i)*X_jt{j,i};
temp(:,:,i) = temp(:,:,i) +beta(j,i)*X_jt{j,i}*X_jt{j,i}';
end
%% 计算后验协方差
Root(i).pkk = P(:,:,i) - (1-beta(1,i))*K(:,:,i)*S(:,:,i)*K(:,:,i)'+temp(:,:,i)-X_t(:,i)*X_t(:,i)';
Root(i).seq =[Root(i).seq X_t(1:2,i)];
Root(i).est =[Root(i).est X_t(:,i)];
Root(i).x_predict = F*X_t(:,i);
end
end

figure
hold on
for j = 1:n
Z = cell2mat(points(j));
plot(Z(1,:),Z(2,:),'.r');
end

for i = 1:t
Root(i).show;
end
xlabel('x轴坐标(m)');
ylabel('y轴坐标(m)');
title('JPDA数据关联');

运行结果如下

JPDA的公式描述看似繁琐但是其实多是在进行二进制的判定,核心仍然是贝叶斯递推,接下来就带着大家进行公式向matlab代码的翻译过程:
首先先简单描述一下代码实现的流程,我尽量用白话和公式各描述一遍,在获得了某一量测的时间点:
语言描述:

  1. 根据上一时刻状态估计协方差进行状态预测协方差以及新息协方差计算,以及测量值的预测值计算
  2. 根据有效的量测以及目标建立确认矩阵
  3. 根据规则对确认矩阵分解为联合事件对应的互联矩阵
  4. 计算每一种联合事件的发生概率
  5. 根据联合事件发生概率计算量测与目标互联的概率
  6. 计算当前时点的目标状态估计
  7. 计算目标状态估计的协方差
    公式描述:(这里使用线性的贝叶斯迭代也就是kalmanfilter)
  8. 式中