HoloOcean:逼真的声纳模拟

HoloOcean: Realistic Sonar Simulation(HoloOcean:逼真的声纳模拟)

期刊信息：2022 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS)（会议）

Abstract

Sonar sensors play an integral part in underwater robotic perception by providing imagery at long distances where standard optical cameras cannot. They have proven to be an important part in various robotic algorithms including localization, mapping, and structure from motion. Unfortunately, generating realistic sonar imagery for algorithm development is difficult due to the high cost of field trials and lack of simulation methods. To remove these obstacles, we present various upgrades to the sonar simulation method in HoloOcean, our open-source marine robotics simulator. In particular, we improve the noise modeling using a novel cluster-based multipath ray-tracing algorithm, various probabilistic noise models, and material dependence. We also develop and integrate simulated models for side-scan, single-beam, and multibeam profiling sonars.
声纳传感器在水下机器人感知中发挥着不可或缺的作用，它提供了标准光学相机无法提供的远距离图像。它们已被证明是各种机器人算法的重要组成部分，包括定位，映射和运动结构。不幸的是，由于现场试验的高成本和缺乏仿真方法，为算法开发生成逼真的声纳图像是困难的。为了消除这些障碍，我们对我们的开源海洋机器人模拟器HoloOcean中的声纳模拟方法进行了各种升级。特别是，我们使用一种新的基于聚类的多路径光线跟踪算法、各种概率噪声模型和材料依赖性来改进噪声建模。我们还开发和集成了侧向扫描、单波束和多波束剖面声纳的模拟模型。

I. INTRODUCTION

Many algorithms for autonomous underwater vehicles (AUVs) leverage the use of sonar imagery to localize, detect objects, generate maps, or perform 3D reconstruction where standard optical imagery can be unusable due to water turbidity. These algorithms are useful for many applications of AUVs such as marine infrastructure inspection, exploration of oceans, and many others. These applications have the potential to drastically increase safety, quality of life, and general scientific knowledge.
许多用于自主水下航行器(auv)的算法利用声纳图像来定位、检测物体、生成地图或执行3D重建，而标准光学图像可能因水浑浊而无法使用。这些算法对auv的许多应用都很有用，例如海洋基础设施检查、海洋勘探等。这些应用有可能极大地提高安全性、生活质量和一般科学知识。

Unfortunately, generating the required realistic sonar imagery for initial algorithm development can be difficult due to field trials being expensive and high-risk. While simulation has the potential to help with this, current simulation methods lack the heavy noise present in real-world sonar imagery. In real-world scenarios, this noise is derived from a variety of sources including, but not limited to, reverberation, inaccurate sensor readings, multi-path acoustic wave propagation, and diffusion.
不幸的是，由于现场试验昂贵且高风险，为初始算法开发生成所需的逼真声纳图像可能很困难。虽然模拟有可能帮助解决这一问题，但目前的模拟方法缺乏真实声纳图像中存在的严重噪声。在现实世界中，这种噪声来自多种来源，包括但不限于混响、不准确的传感器读数、多径声波传播和扩散。

HoloOcean [1] is an open-source underwater simulator built upon Unreal Engine 4 (UE4) [2]. It includes a novel sonar simulation method leveraging an octree representation of the environment as compared to computationally expensive ray-tracing approaches. We have augmented the sonar simulation in HoloOcean with the following features:
HoloOcean[1]是一个基于虚幻引擎4 (UE4)[2]的开源水下模拟器。它包括一种新的声纳模拟方法，利用环境的八叉树表示，而不是计算昂贵的光线追踪方法。我们在HoloOceann中增强了声纳模拟，具有以下功能:

1. A novel cluster-based ray-tracing method to efficiently compute significant multi-path noise found in sonar imagery,
2. Implementation of various probabilistic noise models that more accurately represent reality and produce realistic looking sonar imagery, 3) Easily editable material dependence built into the octree structure, and
3. Implementations of echosounder, side-scan, and multibeam profiling sonars all built upon the octree representation.
   1)一种新的基于聚类的光线跟踪方法，可以有效地计算声纳图像中的重要多径噪声;
   2)实现各种概率噪声模型，更准确地代表现实并产生逼真的声纳图像，
   3)易于编辑的材料依赖内置于八叉树结构中，以及
   4)实现回声测深，侧扫和多波束剖面声纳都建立在八叉树表示之上。

The paper proceeds as follows. Section II reviews current sonar simulation methods and models. In Section III, we describe our novel cluster-based multi-path algorithm, as well as demonstrate our implementation of acoustic reflection models based on material properties. We also describe the various probabilistic models chosen to perturb the data in the same section. This is followed by our side-scan sonar implementation details in Section IV, multibeam profiling sonar in Section V, and the echosounder in Section VI. Finally, Section VII concludes the article and proposes future work. More information about HoloOcean can be found at https://holoocean.readthedocs.io/.
本文的工作如下。第二节回顾了当前的声纳仿真方法和模型。在第三节中，我们描述了新颖的基于聚类的多路径算法，并展示了我们基于材料特性的声反射模型的实现。在同一节中，我们还描述了选择用于扰动数据的各种概率模型。接下来是第四节的侧扫声纳实现细节，第五节的多波束剖面声纳，以及第六节的回声测深仪。最后，第七部分对本文进行总结，并提出今后工作的建议。有关HoloOcean的更多信息，请访问https://holoocean.readthedocs.io/。

II. RELATED WORK

Creating a realistic underwater sonar simulation is a difficult problem, requiring many pieces for the image to be usable as a fill-in for real-world data. These include accurate modeling of the heavy noise often found in sonar imagery, modeling of high fidelity environments, and modeling of the various types of sonar sensors.
创建一个真实的水下声纳模拟是一个难题，需要许多部分的图像是可用的，以填补真实世界的数据。这些包括在声纳图像中经常发现的重噪声的精确建模，高保真环境的建模，以及各种类型的声纳传感器的建模。

Current sonar simulation methods include ray-tracing [3, 4] or GPU-based methods [5], but little work has been done on modeling the heavy noise present in sonar imagery. This noise is a product of multi-path reverberations, acoustic scattering, general sensor noise, ambient waves, and many other phenomena. Many simulation methods model speckle noise using additive and multiplicative methods, but ignore other forms of noise [6, 7]. Attempts have been made to leverage a generative adversarial network [8, 9] or style transfer networks [10] to recreate this noise, but those lack a clear way to extrapolate to more complex environments or between varying sonar parameters.
目前的声纳仿真方法包括光线追踪[3,4]或基于GPU的方法[5]，但对声纳图像中存在的重噪声进行建模的工作很少。这种噪声是多径混响、声散射、一般传感器噪声、环境波和许多其他现象的产物。许多仿真方法使用加性和乘法方法对散斑噪声建模，但忽略了其他形式的噪声[6,7]。已经有人尝试利用生成对抗网络[8,9]或风格转移网络[10]来重建这种噪音，但这些都缺乏明确的方法来推断更复杂的环境或不同声纳参数之间的情况。

Under our own observation, one of the primary noises present in sonar imagery is largely due to multi-path reflections off of objects or structures in the environment. This noise is difficult to recreate in real time due to the heavy computational cost. Preliminary attempts to selectively raytrace multi-path reverberations have been done using a virtual camera [11], but are limited due to the camera field of view not matching that of a true sonar.
根据我们自己的观察，声纳图像中存在的主要噪声之一主要是由于环境中物体或结构的多路径反射。由于计算成本高，这种噪声很难实时再现。已经使用虚拟摄像机进行了选择性光线追踪多径混响的初步尝试[11]，但由于摄像机的视场与真实声纳的视场不匹配，因此受到限制。

The intensities measured by sonars are dependent on properties of the materials the acoustic waves reflected off of, specifically acoustic impedance that depends on the material speed of sound and material density [12]. While this acoustic impedance has been included in sonar simulations previously [4, 13], it has generally been in simple environments with limited material variety.
声呐测量的强度取决于声波反射的材料的特性，特别是声阻抗，这取决于材料的声速和材料密度[12]。虽然这种声阻抗先前已被包括在声纳模拟中[4,13]，但它通常是在材料种类有限的简单环境中。

Work has been done in simulating a side-scan sonar [14– 17], profiling sonar [18], and to a lesser extent echosounder [19]. However, these approaches are all spread out across numerous simulators, are generally ray-tracing-based methods that can be computationally expensive, and are often not open-source.
在模拟侧扫声纳[14 - 17]，剖面声纳[18]以及较小程度的回声测深仪[19]方面已经完成了工作。然而，这些方法都分散在许多模拟器中，通常是基于光线跟踪的方法，计算成本很高，而且通常不是开源的。

HoloOcean [1] has an octree-based sonar simulation method, allowing for fast simulation by avoiding ray-tracing, and easy access to the structure of the environment. We augment its simulation method by adding in a clusteringbased multi-path propagation algorithm; material dependence in large, complex UE4-based environments; various probabilistic models for noise introduction; and implementations of the sidescan, profiling, and echosounder sonars.
HoloOcean[1]有一种基于八叉树的声纳模拟方法，可以通过避免光线追踪来实现快速模拟，并且易于访问环境的结构。我们通过加入基于聚类的多路径传播算法来增强其仿真方法;基于UE4的大型复杂环境中的材料依赖;噪声引入的各种概率模型;以及侧扫、剖面和回声测深声纳的实现。

III. MODELING NOISE IN IMAGING SONAR （成像声纳中的噪声建模）

A multibeam imaging sonar is a common sensor in underwater robotics used for generating imagery of the environment. It generates these images by sending out acoustic pings, which upon return are categorized by their intensity, range that is calculated from the time of flight, and azimuth angle that is estimated via beamforming methods. Results are used to create a 2D image where each pixel value is the intensity for a range and azimuth interval. This geometry is shown in Fig. 2(a).
多波束成像声纳是水下机器人中常用的传感器，用于生成环境图像。它通过发出声波脉冲来生成这些图像，这些声波脉冲在返回时根据其强度、从飞行时间计算的范围以及通过波束形成方法估计的方位角进行分类。结果用于创建二维图像，其中每个像素值是范围和方位角间隔的强度。这种几何结构如图2(a)所示。

Fig. 2: Geometric representation of various sonar types. (a) Geometry of a multibeam imaging sonar, where the intensities are projected onto the range r, azimuth ϕ plane and elevation θ is lost. A multibeam profiling sonar functions similarly, with the exception that the elevation θ is so tight it’s negligible. (b) Side-scan sonar geometry, where elevation angle is negligible, and the intensities are projected onto the r axis, and then concatenated. © Cone shape of an echosounder ping, where intensities are projected onto the r axis, and semi-vertical angle ψ is lost. Central angle τ is used for shadowing purposes.
图2:各种声纳类型的几何表示。(a)多波束成像声纳的几何形状，其中强度投影到距离r，方位φ平面和仰角θ丢失。多波束剖面声纳的功能类似，除了仰角θ太紧，可以忽略不计。(b)侧扫声纳几何，俯仰角可以忽略不计，并将强度投影到r轴上，然后拼接。©回声探测仪ping信号的圆锥形状，其中强度被投射到r轴上，并且丢失了半垂直角ψ。圆心角τ用于遮蔽目的。

A. Simulation Model （仿真模型）

HoloOcean’s sonar simulation method is based upon an octree structure of a UE4 environment [1]. At each timestep, this octree is recursively searched for all the leaves that are in the field of view. These leaves are then sorted in azimuth ϕ, elevation θ bins which are then sorted by range in ascending order. The leaves beyond the first cluster are removed as they are in shadows, and all the resulting leaves have their intensities calculated using surface normals. Intensities are then averaged over each r, ϕ bin. We have built on this simulation model by adding multi-path contributions and various noise improvements, as explained in the following subsections.
HoloOcean的声纳仿真方法基于UE4环境的八叉树结构[1]。在每个时间步，这个八叉树递归地搜索视野中的所有叶子。这些叶然后排序在方位角$\phi$，海拔$\theta$箱，然后排序的范围，以升序。第一个集群之外的叶子被移除，因为它们在阴影中，并且所有产生的叶子都使用表面法线计算其强度。然后将强度平均到每个$r,\varphi$ bin上。我们通过添加多路径贡献和各种噪声改进来构建此仿真模型，如下面的小节所述。

B. Multi-path (多路径)

Building upon the octree structure that HoloOcean provides, after locating and sorting the octree leaves in the sonar field of view into r, ϕ, θ bins, multi-path contributions are then ready to be computed. Note each leaf has unit normal n, location p represented in the sonar frame, and a return unit vector v pointing to the sonar stored as part of its structure, all of which are used throughout. We denote which leaf these belong to through their subscript, for example leaf k has values nk, pk, vk.
基于HoloOcean提供的八叉树结构，将声纳视场中的八叉树叶子定位并分类为$r,\varphi ,\theta$bin后，就可以计算多路径贡献了。注意，每个叶子都有单位法向$n$，位置$p$在声纳帧中表示，返回单位向量$v$指向声纳作为其结构的一部分存储，所有这些都贯穿始终。我们通过它们的下标来表示它们属于哪个叶子，比如叶子$k$的值是$n_k,p_k,v_k$。

First, each leaf is sorted into a cluster. Clusters are made 12by choosing an octree leaf o and selecting all corresponding leaves within ϵ bins that have normals that are within δ degrees of the original leaf. In practice, we have found values of ϵ = 5, δ = 15◦ have given good results.
首先，每个叶子被分类到一个簇中。聚类是通过选择一个八叉树叶子，并在与原始叶子的正交度在δ度以内的λ箱中选择所有相应的叶子来形成的。在实践中，我们发现$\epsilon =5，\delta =15^{◦}$的值给出了很好的结果。

The original leaf is then ray-traced. This is done by rotating the return vector vo by 180◦ about its normal no as follows [20]
然后对原始叶子进行光线追踪。这是通过将返回向量$v_o$绕其法向$n_o$旋转$180^{◦}$来实现的，如下所示[20]
$$r_0=-v_0+2(n_0\cdot v_0)n_0(1)$$
We then step along the bins in the direction of $r_o$ until either we step out of the field of view of the sonar or hit another leaf $h$.
然后我们沿着箱子朝 $r_o$ 方向前进，直到我们走出声纳的视野，或者撞到另一片叶子$h$。

If we hit a leaf h, we make the assumption that the small region about h is planar and defined by the location ph, normal nh, and return vector vh. This assumption implies that any leaf i in the cluster will bounce off this same plane. Ray-tracing i then simplifies to finding the intersection between the line given by pi , ri and the plane ph, nh reducing computations significantly. A closed form solution is given by [21]
如果我们碰到一个叶子$h$，我们假设$h$周围的小区域是平面的，由位置$p_h$，法向$n_h$和返回向量$v_h$定义。这个假设意味着集群中的任何叶子i都会在同一平面上反弹。然后，射线追踪i简化为寻找由$p_i,r_i$给出的直线与平面$p_h,n_h$之间的交点，大大减少了计算量。一个封闭解由[21]给出
$$p=p_i+r_i(\frac{(p_h-p_i)\cdot n_h}{r_i\cdot n_h})(2)$$
These steps are all visualized in Fig. 3. From this p, a final azimuth angle can be computed, as well as the resulting distance as follows
这些步骤都在图3中可视化。从这个p，可以计算出最终的方位角，以及如下所示的最终距离
$$d=\frac{||p_i||+||p-p_i||+||p_i||}{2}(3)$$

Fig. 3: Visualization of cluster-based multi-path algorithm. A cluster C is created around an octree leaf o, which has normal, position, reflection, and return vector of no, po, ro, vo, respectively. Leaf o is ray-traced to leaf h along ro, where when impact is found −ro is reflected into q. The surface at leaf h is then approximated using a plane created from nh, ph, off of which the remaining leaves in cluster C will be reflected.
图3:基于聚类的多路径算法可视化。集群C是围绕八叉树叶子o创建的，它的法线、位置、反射和返回向量分别为$n_o、p_o、r_o、v_o$。叶子$o$沿着$r_o$被光线追踪到叶子$h$，当发现碰撞时- $r_o$被反射到$q$。然后使用由$n_h,p_h$创建的平面来近似叶子$h$的表面，在此基础上，簇C中剩余的叶子将被反射。

We compute q by reflecting −ri off of nh as in eq. (1). Using q, the resulting intensity value is found by dotting the return ray of p with q, and scaling by both materials’ power reflection coefficients as in Section III-C. This intensity value is then averaged along with all the other leaf and multipath contributions in the same r, ϕ bin. These steps are summarized in Fig 4.
我们通过从$n_h$反射$-r_i$来计算$q$，如式(1)所示。使用$q$，通过将$p$的返回射线与$q$点乘，并按两种材料的功率反射系数缩放，如第III-C节所示，得到的强度值。然后将该强度值与相同$r,\varphi$ bin中的所有其他叶子和多路径贡献一起平均。图4总结了这些步骤。

Fig. 4: Pseudocode for clustering-based multi-path algorithm. Lines 4-10 correspond to the actual basic clustering algorithm, lines 11-14 to the raytracing of a single cluster element, and lines 15-21 to the ray-tracing of the rest of the cluster.
图4:基于聚类的多路径算法伪代码。第4-10行对应于实际的基本聚类算法，第11-14行对应于单个聚类元素的光线跟踪，第15-21行对应于聚类其余部分的光线跟踪。

C. Material Dependence(材料依赖)

The intensities measured by the sonar are dependent on the materials that the acoustic waves reflected off of. Given normal incidence, the power reflection coefficient off leaf k is given by [12]
材料依赖声呐测量的强度取决于声波反射的材料。给定正入射，叶片$k$的功率反射系数为[12]
$$\begin{gathered} R_{II}=\frac{I_r}{I_i}=(\frac{z_k-z_w}{z_k+z_w}{)}^2(4) \\ {z}_k=c_k\cdot {\rho }_k(5) \end{gathered}$$
where $I_r,I_i$ are the reflected and incident intensities respectively; $z_k,z_w$ the specific acoustic impedance of leaf $k$ and water respectively; and ${\rho }_k,c_k$ are the density and the speed of sound, respectively, of leaf $k$.
式中$I_r,I_i$分别为反射强度和入射强度;$z_k,z_w$ 分别为叶片k和水的比声阻抗${\rho }_k,c_k$分别是叶片$k$的密度和声速。

Using this relation, we have the following approximation for the returned intensity of leaf $k$ as follows
利用这个关系，我们得到叶片$k$返回强度的近似如下
$$I_k=(\frac{z_k-z_w}{z_k+z_w}{)}^2(v_k\cdot n_k)(6)$$
where $I_k$ is the return intensity, $v_k$ the return vector, and $n_k$ the surface unit normal all of the octree leaf $k$. Note $v_k\cdot n_k$ results in the angle between the two unit vectors.
其中$I_k$是返回强度，$v_k$是返回向量，$n_k$是八叉树叶子$k$的表面单位法向。注意$v_k\cdot n_k$ 的结果是两个单位向量之间的夹角。

Material names are pulled directly from UE4 and the material properties ${\rho }_k,c_k$ for each material are saved in a csv file for easy editing without the need for recompilation. This allows for on-the-fly tweaks to materials’ properties between simulation missions.
材质名称直接从UE4中提取，材质属性${\rho }_k,c_k$为每个材质保存在一个csv文件中，便于编辑，无需重新编译。这允许在模拟任务之间对材料属性进行动态调整。

D. Probabilistic Contributions (概率的贡献)

Apart from the noise contributed by multipath, A number of other sources of noise are present in sonar imagery. We chose to perturb our imagery using various distributions to introduce these noise sources.
除了多路径产生的噪声外，声纳图像中还存在许多其他噪声源。我们选择使用不同的分布来干扰我们的图像，以引入这些噪声源。

The time of flight is measured by the sonar and is proportional to the reflection distance. This measured value often suffers from diffusion, meaning it may result in some returns longer than the actual time of flight, but none shorter since that would be physically impossible. This phenomenon has previously been modeled with a K-distribution [23], but for computational efficiency, we approximately perturb $r$ with the similar, easy to sample from exponential distribution
飞行时间由声纳测量，并与反射距离成正比。这个测量值经常受到扩散的影响，这意味着它可能会导致一些返回时间长于实际飞行时间，但不会比实际飞行时间短，因为这在物理上是不可能的。这种现象以前已经用K分布建模[23]，但为了计算效率，我们近似地用类似的、易于从指数分布中采样的扰动$r$
$$\tilde{r}=r+w^r,(7)$$

Not only is the time of flight perturbed by the exponential distribution, but the intensity returns diminish the more $r$ is perturbed. We model this by scaling the intensity using a scaled probability density function $f_{W^r}$
不仅飞行时间受指数分布的干扰，而且$r$受干扰越大，强度回报越小。我们通过使用缩放概率密度函数$f_{W^r}$缩放强度来建模
$$\widetilde{I_k}=I_k{f}_{W^r}(w^r)\lambda =I_{k}exp(\frac{-w^r}{\lambda })(8)$$
These intensities are then averaged over all leaves in a bin to produce a single pixel intensity,
然后将这些强度平均到一个箱子中的所有叶子上，以产生单个像素强度，
$$I_{ij}=\frac{1}{n}\sum _{k=0}^n{\tilde{I}}_k.(9)$$

These resulting intensities are often scaled onboard the sonar to produce uniform pixel intensities. Since acoustic intensity is proportional to $\frac{1}{r^2}$ [12], it is often normalized on board the sonar using $r^2$ to get uniform pixel density. This processing method has the effect of increasing the magnitude of the noise as range increases. Also, the lobe shape of the transducer is often used as a normalization factor. Since our intensity is already normalized, these scale factors will only influence the speckle noise. We thus modify the additive noise $w^{sa}$
这些产生的强度通常在声纳上进行缩放以产生均匀的像素强度。由于声强与$\frac{1}{r^2}$成正比[12]，因此声强在声纳上通常使用$r^2$进行归一化，以获得均匀的像素密度。这种处理方法具有随着距离增大而增大噪声幅度的效果。此外，换能器的瓣形通常用作归一化因子。由于我们的强度已经归一化，这些比例因素只会影响散斑噪声。因此，我们修改了加性噪声$w^{sa}$
$$\begin{gathered} {\tilde{w}}^{sa}=\frac{r_{ij}^2}{r_{max}^2}(1+0.5exp(-{\varphi }_{i}j))w_{sa}(10) \\ {w}^{sa}\sim R({\sigma }^{sa})(11) \end{gathered}$$
where ${\varphi }_{ij},r_{ij}$ are the azimuth and range of the $i,j$ bin; $r_{max}$ is the maximum range of the sonar; and $R$ is the Rayleigh distribution. This additive noise is then combined with multiplicative noise $w^{sm}$
其中，${\varphi }_{ij},r_{ij}$为$i,j$ bin的方位角和范围;$r_{max}$为声呐最大探测距离;R是瑞利分布。然后将该加性噪声与乘性噪声$w^{sm}$相结合，
$$\begin{gathered} {\tilde{I}}_{ij}=I_{ij}(0.5+w^{sm})+w^{sa}(12) \\ {w}^{sm}\sim N(0，{\sigma }^{sm})(13) \end{gathered}$$

E. Azimuth Streaking (方位条纹)

Occasionally, a sonar can receive too many reflections from normal incidences with the same time of flight and make the beamforming problem ill-conditioned. In these scenarios, sonars often return intensities at all azimuth angles for a given range, resulting in an azimuth streak. Some sonars attempt to subtract out these differences, but in doing so can cause more artifacts. We estimate when this ill-conditioning occurs by checking if a certain percentage of leaves found in a given range interval with near parallel n and v exceeds a given threshold $\zeta$. This range interval is then modified as follows to introduce the azimuth artifact,
偶尔，声纳会在相同的飞行时间内接收到过多的正常入射反射，从而使波束形成问题变得病态。在这些情况下，声纳通常会在给定范围内返回所有方位角的强度，从而导致方位角条纹。一些声纳试图消除这些差异，但这样做会导致更多的伪影。我们通过检查在给定范围内$n$和$v$接近平行的叶子的一定百分比是否超过给定阈值$\zeta$来估计这种不良条件何时发生。然后对这个距离间隔进行如下修改以引入方位伪影:
$${\hat{I}}_{ij}=1-(1-{\tilde{I}}_{ij}{)}^2(14)$$
and if the sonar has on-board gain reduction as follows instead,
如果声纳的机载增益降低如下，
$${\hat{I}}_{ij}={\tilde{I}}_{ij}^2(15)$$

An example of the resulting image with the contributions of multi-path, diffusion, additive noise scaling, and removal of azimuth streaks can be seen in Fig. 5. Our approach generates approximately 12 images per second at ranges of 10m in the tank environment.
图5是多路径、扩散、加性噪声缩放和方位角条纹去除的结果图像示例。我们的方法在坦克环境中每秒产生大约12张10米范围内的图像。

Fig. 5: Comparison of real world and simulation sonar imagery taken in our testing tank. (a) Comparison of our real tank, and the UE4 environment of our tank. (b) Sonar image taken from a Blueprint Subsea Oculus M1200d [22] on the HF setting with a 60◦ azimuth, 12◦ elevation, and 10m max range and © our simulation method with identical parameters and a cluster size of ϵ = 5. The multi-path differences in (b) and © at the 9m marks are likely due to differences in sonar pose between real world and simulation. (d) Computation steps of our model from (1) initial octree image, (2) multi-path, (3) range diffusion, (4) additive and multiplicative noise, (5) scaling the additive noise, and finally the azimuth artifact is shown in © at about the 8m mark. The figures in (d) are saturated to more easily visualize the changes.
图5:在我们的测试箱中拍摄的真实世界和模拟声纳图像的比较。(a)我们真实坦克的对比，和我们坦克的UE4环境。(b)从Blueprint Subsea Oculus M1200d[22]中获取的声纳图像，在HF设置下，60◦方位角，12◦仰角，最大距离为10米;©我们的模拟方法具有相同的参数和聚类大小为λ = 5。(b)和©在9米标记处的多路径差异可能是由于真实世界和模拟之间声纳姿态的差异。(d)我们的模型从(1)初始八叉树图像，(2)多路径，(3)距离扩散，(4)加性和乘性噪声，(5)缩放加性噪声，最后©显示在大约8m标记处的方位角伪影的计算步骤。(d)中的数字是饱和的，以便更容易地看到变化。

IV. SIDE-SCAN SONAR (侧扫声纳)

There are other types of sonar sensors that are also common in underwater robotics, such as the side-scan sonar. In order to increase the data generation capabilities of HoloOcean, we have leveraged the existing robust octree structure to simulate side-scan sonar as well.
还有其他类型的声纳传感器在水下机器人中也很常见，比如侧面扫描声纳。为了提高HoloOcean的数据生成能力，我们利用现有的鲁棒八叉树结构来模拟侧扫声纳。

A. Operation (操作)

Side-scan sonars also send out acoustic waves to capture imagery of their environment. They generally operate by pointing down at the seafloor and emitting two separate pulses, one off each side of the sensor. Upon reflection and return, time of flight and intensity are both measured. In contrast to the imaging sonar sensor, angle of arrival is not estimated, so no azimuth is measured. These pulses are generally sent over a large azimuth angle and over a much tighter elevation. This results in a known range and elevation, but unknown azimuth.
侧边扫描声纳也会发出声波来捕捉周围环境的图像。它们通常通过指向海底并发射两个单独的脉冲来工作，每个脉冲来自传感器的两侧。在反射和返回时，测量飞行时间和强度。与成像声纳传感器相比，不估计到达角，因此不测量方位角。这些脉冲通常在一个大的方位角和一个更小的高度上发送。这导致已知的范围和高度，但未知的方位角。

The resulting intensities are then binned from max range to 0m on the left and appended to the array of 0m to max range on the right [17]. A blank portion always occurs in the middle of this data since no objects exist within a certain distance of the sonar.
然后将得到的强度从左边的最大范围到0米进行分类，并将其附加到右边的0米到最大范围的数组中[17]。由于在声纳的一定距离内没有物体存在，因此这些数据的中间总是出现空白部分。

When visualizing side-scan imagery, each time-step is stitched together to form a waterfall plot; where the y-axis denotes the time-step of the ping, x-axis is the range from max range on the left to 0 in the center to max range on the right, and each pixel value is the given intensity. This geometry is summarized in Fig. 2(b).
在可视化侧面扫描图像时，每个时间步被拼接在一起形成瀑布图;其中y轴表示ping的时间步长，x轴表示从左边的Max range到中间的0再到右边的Max range的范围，每个像素值是给定的强度。这种几何结构如图2(b)所示。

B. Simulation Model （仿真模型）

The side-scan sonar simulation model is built upon the existing octree structure and the imaging sonar implementation including the existing recursive leaf searching method, shadowing algorithm, and noise models.
侧扫声纳仿真模型建立在现有的八叉树结构和成像声纳实现的基础上，包括现有的递推叶搜索方法、阴影算法和噪声模型。

It does differ in how the found leaves are sorted. If $\varphi <0$, a leaf is added to the right beam, and to the left beam if $\varphi >0$. The left beam’s leaves are binned from largest to smallest range on the left, and the right beam’s leaves from smallest to largest range. These beams are then concatenated, and the resulting intensity computed as in eqs. (6) and (9). Resulting imagery can be seen in Fig. 7.
不同的是找到的叶子是如何排序的。如果$\varphi <0$，则将叶片添加到右光束，如果$\varphi >0$则添加到左光束。左光束的叶子在左边从最大到最小的范围被分类，右光束的叶子从最小到最大的范围被分类。然后将这些光束连接起来，并按公式计算得到的强度。(6)和(9)。得到的图像如图7所示。

Fig. 7: Example of output of the side-scan sonar simulation. This was taken over 1600 timesteps, each of which results in a row of the image.
图7:侧扫声纳仿真输出示例。这需要1600多个时间步，每个时间步产生一行图像。

V. MULTIBEAM PROFILING SONAR (多波束剖面声纳)

For mapping large areas of the seafloor, a multibeam profiling sonar is used for generating low resolution bathymetry maps quickly. Due to its excessive size, weight, and high power consumption, profiling sonars are typically mounted to surface vessels instead of AUVs. The sonar is mounted to the underside of the vehicle pointing downwards towards the seafloor where it scans a broad sweep of the seafloor directly beneath. A systematic survey pattern can then be used for mapping larger areas.
对于大面积的海底测绘，多波束剖面声纳用于快速生成低分辨率的海底测深图。由于剖面声纳的尺寸、重量和功耗过大，通常安装在水面舰艇上，而不是auv上。声纳安装在潜水器的底部，向下指向海底，在那里它可以扫描直接下方的海底。然后，系统的调查模式可以用于绘制更大的区域。

A. Operation(操作)

The sonar is comprised of multiple physical sensors, called a transducer array, that send and receive sound pulses that reflect off the seafloor or other present objects. Similar to the imaging sonar, the profiling sonar has a wide azimuth angle that allows a wide field of view, but differs in that the elevation angle is much narrower. With a narrow elevation angle, typically 1 to 3 degrees, the profiling sonar can estimate the true distance and return an array of measurements of the environment scanned below. While surveying an area, by saving the sonar measurements with the estimated pose of the robot at each time step, a 3D representation of the environment can be created from these 2D slices.
声纳由多个物理传感器组成，称为传感器阵列，可以发送和接收海底或其他物体反射的声音脉冲。与成像声纳类似，剖面声纳具有宽方位角，允许宽视野，但不同之处在于仰角窄得多。通过一个狭窄的仰角，通常为1到3度，剖面声纳可以估计真实距离，并返回下面扫描环境的一系列测量值。在测量一个区域时，通过保存声纳测量数据和机器人在每个时间步的估计姿态，可以从这些2D切片中创建环境的3D表示。

B. Simulation Model (仿真模型)

Due to the similar characteristics of the imaging sonar, the profiling simulation model implementation is nearly identical, utilizing the recursive leaf searching method and noise models. The models differ in that the profiling only utilizes one elevation bin for shadowing. To demonstrate the use of the profiling sonar in simulation, a simple mission was planned gathering bathymetry data shown in Fig. 6. An example profiling sonar image is shown, along with extracted x, y coordinates, and a resulting bathymetry map.
由于成像声纳的相似特性，采用递推叶搜索法和噪声模型实现了几乎相同的剖面仿真模型。模型的不同之处在于，剖面只利用一个高程仓进行阴影。为了演示剖面声纳在模拟中的使用，计划了一个简单的任务，收集测深数据，如图6所示。显示了一个示例剖面声纳图像，以及提取的x, y坐标和由此产生的测深图。

Fig. 6: Example of multibeam sonar simulation and uses. (a) Example of simulated multi-beam sonar imagery taken over an ocean floor looking down. (b) The x,y coordinates extracted from (a), and © 120 of these scans placed side by side to create a bathymetry map. Note the techniques used to extract x,y coordinates were rather simple, and the added noise naturally impacted the results. A more advanced bathymetry algorithm is beyond the scope of this paper.
图6:多波束声纳仿真及使用示例。(a)在海底向下看的模拟多波束声纳图像示例。(b)从(a)和© 120次扫描中提取的x,y坐标并排放置，以创建测深图。注意，用于提取x、y坐标的技术相当简单，添加的噪声自然会影响结果。一种更先进的测深算法超出了本文的范围。

VI. ECHOSOUNDER (回声探测器)

Echosounders, or single-beam sonar sensors, are common systems used for determining water depth, which is useful for mapping and navigation. Echosounders are often cheaper than other sonars and have wide field of view condensed in a single beam. Although echosounders do not produce maps as detailed as those from a multibeam sonar, they can detect schools of fish and even plankton. They can also be used to prevent collisions with large objects beneath ships. Similar to the multibeam sonar, echosounders are usually mounted to the bottom of vehicles pointed towards the seafloor or on the nose of an AUV pointing forward. However, single-beam sonars take longer to map an area than a multibeam sonar because the resolution of the data depends on the speed of the vehicle on which the sonar is mounted.
回声测深仪或单波束声纳传感器是用于确定水深的常用系统，这对测绘和导航很有用。回声探测仪通常比其他声纳便宜，并且在单一波束中有广阔的视野。虽然回声探测仪不能像多波束声纳那样绘制出详细的地图，但它们可以探测到鱼群，甚至浮游生物。它们还可以用来防止与船下的大型物体相撞。与多波束声纳类似，回声探测仪通常安装在指向海底的车辆底部，或者安装在指向前方的水下航行器的前端。然而，单波束声纳要比多波束声纳花更长的时间来绘制一个区域的地图，因为数据的分辨率取决于安装声纳的车辆的速度。

A. Operation (操作)

The echosounder generally uses a single transducer to transmit and receive acoustic signals in the shape of a cone. Because the speed of sound in water is known, echosounders can use the round-trip time of a signal to calculate the distance traveled. This distance can then be divided in two to determine the distance from the boat to the seafloor, and returned in an array of intensities where each array element corresponds to a different range interval. This geometry is shown in Fig. 2©.
回声探测仪通常使用单个换能器来发送和接收锥形的声学信号。因为声音在水中的传播速度是已知的，回声探测仪可以利用信号的往返时间来计算传播的距离。然后可以将这个距离分成两部分，以确定从船到海底的距离，并以强度数组返回，其中每个数组元素对应于不同的范围间隔。这种几何结构如图2©所示。

B. Simulation Model(仿真模型)

The echosounder simulation also heavily inherits from the previously mentioned imaging sonar sensor in terms of octree leaf searching and shadowing. However, because of the inherent cone shape of the echosounder, it does not have an elevation or azimuth angle. Instead, only the range and semi-vertical angle ψ of the cone are used to determine if a leaf is within the field of view of the sonar. An additional angle, which we will call the central angle τ , is also used to create bins for shadowing purposes.These angles are visualized in Fig. 2©. The signal intensities are sorted into ψ, τ bins, and shadowing is computed with these bins in an identical manner as the imaging sonar ϕ, θ bins, as described in Subsection III-A. After shadowing, the remaining leaves are sorted into range bins, the intensities in each bin are averaged, and then returned.
回声测深仿真在八叉树叶搜索和阴影方面也大量继承了前面提到的成像声纳传感器。然而，由于回声探测仪固有的圆锥形状，它没有仰角或方位角。相反，只有锥体的距离和半垂直角ψ被用来确定一片叶子是否在声纳的视野范围内。一个额外的角度，我们称之为圆心角τ，也用于创建用于阴影目的的箱。这些角度如图2©所示。信号强度被分类到ψ， τ箱中，阴影用这些箱以与成像声纳φ， θ箱相同的方式计算，如第III-A小节所述。遮蔽后，将剩余的叶子分类到范围箱中，对每个箱中的强度取平均值，然后返回。

The range calculations are perturbed using the exponential distribution, similar to the imaging sonar case. The intensities are not scaled by the probability density function though, as there is little intensity drop-off for this diffusion.
距离计算使用指数分布进行扰动，类似于成像声纳的情况。然而，强度不是由概率密度函数缩放的，因为这种扩散的强度几乎没有下降。

Fig. 8 shows comparison of real-world data and our simulation model, with an opening angle of 30 degrees, or a semi-vertical angle of 15 degrees.
图8显示了真实数据与我们的仿真模型的比较，开口角为30度，半垂直角为15度。

VII. CONCLUSION （结论）

Based on the octree simulation techniques developed in our previous work [1], we augment HoloOcean, our opensource marine robotics simulator, with a side-scan, multibeam profiling, and echosounder sonar implementations.
These are all built upon the existing octree structure to provide realistic imagery of the complex environments in UE4 engine. Further, we improve the noise modeling through a cluster-based ray-tracing method for multi-path generation, as well as various probabilistic contributions for bin diffusion, additive noise scaling, material dependence, and azimuth streaking. The resulting imagery is realistic and can be used for underwater algorithm development. Future work includes a GPU implementation of the sonar algorithms, additional agents including autonomous surface vessels, and additional custom environments.
基于我们之前的工作[1]中开发的八叉树仿真技术，我们增强了HoloOcean，我们的开源海洋机器人模拟器，具有侧扫描，多波束剖面和回声测深声纳实现。这些都是建立在现有的八叉树结构，以提供逼真的图像在UE4引擎的复杂环境。此外，我们通过基于聚类的光线跟踪方法改进了多路径生成的噪声建模，以及bin扩散、加性噪声缩放、材料依赖和方位角条纹的各种概率贡献。所得到的图像是真实的，可以用于水下算法的开发。未来的工作包括声纳算法的GPU实现，包括自主水面舰艇在内的其他代理，以及额外的自定义环境。