GIS

如何通过GPS坐标查询Adcode

最近业务需要通过GPS坐标拿到坐标所在城市Adcode的信息,这个功能之前系统有一版是通过百度的POI查询接口实现的。由于百度POI查询接口每天有限额,而业务也可以通过在上报坐标时带上Adcode来规避查询,所以当时让业务调整方案不查询Adcode。
这个需求并不难处理,行政区划数据很容易就能获取到,比如可以从下面的地址找到下载地址:
https://www.jianshu.com/p/cb3ccbae9c88
第一步 我们下载行政区划数据,当然也可能调用地图服务商提供的接口来查询行政区划的数据,比如高德的行政区划接口,它把数据分成四级(国、省、市、区)。
第二步 整理数据,把数据分为省、市、区,对于省和市关联上对应的区划边界,比如:
adcode name center level 边界
650100 乌鲁木齐市 87.6177,43.7928 市 xxx,xxx;xxx,xxx
650000 新疆维吾尔自治区 87.6177,43.7928 省 xxx,xxx;xxxx,xxxx
第三步 考虑如何实现查询,主要考虑了三个方案PostGis,MySQL,JTS库自实现;
PostGis是空间数据库,它是通过向PostgreSQL添加空间数据类型、空间索引和空间函数的支持,将PostgreSQL转化空间数据库。它几乎可以称的上是GIS应用数据库的标准实现了。PostGis空间数据库插入空间数据时要建立空间索引,性能会受到索引插入的影响,适合不经常变动的数据,比如地图、热点POI数据;如果是人车的实时位置就不合适了。
MySQL也提供了空间函数和类型的支持;按下面的语句建立数据表,插入区划数据,就可以很方便的使用了,试着查了下,性能还不错
查询

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
CREATE TABLE `t_district` (
  `id` BIGINT(21) NOT NULL AUTO_INCREMENT,
  `adcode` VARCHAR(6) COLLATE utf8_bin NOT NULL,
  `name` VARCHAR(32) COLLATE utf8_bin NOT NULL,
  `center` VARCHAR(64) COLLATE utf8_bin NOT NULL,
  `level` tinyint(2) NOT NULL COMMENT '级别,国家、省/直辖市、市、区/县4个级别) 可选值:0、1、2、3',
  `polyline` geometry NOT NULL COMMENT '边界',
  `parent` BIGINT(21) NOT NULL,
  `create_time` BIGINT(21) NOT NULL DEFAULT '0' COMMENT '记录创建时间',
  `update_time` BIGINT(21) NOT NULL DEFAULT '0' COMMENT '修改时间',
  PRIMARY KEY (`id`),
  KEY `idx_adcode` (`adcode`),
  KEY `idx_level` (`level`),
  SPATIAL KEY `idx_polyline` (`polyline`)
) ENGINE=InnoDB AUTO_INCREMENT=371 DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
-- 插入数据
INSERT INTO `t_district`(`adcode`,`name`,`center`,`level`,`polyline`) 
VALUES ('100000','其它','116.3683244,39.915085','2',ST_GEOMFROMTEXT('POLYGON((xx xx,xx xx),(xx xx)))) 
...
-- 查询
SELECT  id, adcode, name, center,  level, parent, create_time, update_time
FROM t_district
WHERE ST_CONTAINS(polyline, ST_GEOMFROMTEXT('POINT(116.485684 39.921122)'))

JTS库实现,只需要在项目中引入JTS依赖库,性能在MAC本上最差情况下10000的qps轻轻松松,方案简单也不用运维PostGIS和MySQL。我们详细讨论一下如何使用JTS库实现行政区划查询
第三步 确定方案,使用JTS库自已实现,项目引入JTS库

1
2
3
4
5
<dependency>
    <groupId>com.vividsolutions</groupId>
    <artifactId>jts</artifactId>
    <version>1.13</version>
</dependency>

对每个省、城市生成区划多边形数据,每个省市的边界数据放到txt文件,原始数据格式为:lon1,lat1;lon2,lat2|lon3,lat3….
例如:

1
116.556408,34.012041;116.556408,34.012041;116.556408,34.012041|116.556408,34.012041

数据文件命名规则如下,部署时,数据文件和程序代码是分开存储的

1
2
3
4
5
6
130300_city.txt
130400_city.txt
130500_city.txt
130600_city.txt
130700_city.txt
140000_province.txt

读文件,构造Adcode与边界数据的映射Map

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
/**
 * 遍历边界文件,维护adcode与polyline的映射
 */
private static class Visitor extends SimpleFileVisitor<Path> {
 
    private Map<String, String> map;
 
    public Visitor(Map<String, String> map) {
        this.map = map;
    }
 
    @Override
    public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
        String fileName = file.getFileName().toString();
        String adcode = fileName.substring(0, fileName.indexOf('_'));
        List<String> strings = java.nio.file.Files.readAllLines(file);
        this.map.put(adcode, strings.get(0));
        return super.visitFile(file, attrs);
    }
}
// 定义常量
private static volatile ImmutableMap<String, DistrictPolyline> CITY_POLYLINE_MAP;
private static GeometryFactory FACTORY = new GeometryFactory();
// 读取文件
Path path = Paths.get(DATA_PATH + "/polyline");
Map<String, String> polylinesInStr = Maps.newLinkedHashMap();
java.nio.file.Files.walkFileTree(path, new Visitor(polylinesInStr));

构建CITY_POLYLINE_MAP

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Map<String, DistrictPolyline> map = Maps.newLinkedHashMap();
CITY_ADCODE.forEach(adcode -> {
    String line = polylinesInStr.get(adcode);
    String[] polylineStrs = line.split("\\|");
    Polygon[] polygons = new Polygon[polylineStrs.length];
    for (int j = 0; j < polylineStrs.length; j++) {
        String s = polylineStrs[j];
        String[] points = s.split(";");
        Coordinate[] cs = new Coordinate[points.length];
        for (int i = 0; i < points.length; i++) {
            String p = points[i];
            String[] lonLat = p.split(",");
            cs[i] = new Coordinate(Double.parseDouble(lonLat[0]), Double.parseDouble(lonLat[1]));
        }
        LinearRing ring = FACTORY.createLinearRing(cs);
        Polygon polygon = FACTORY.createPolygon(ring);
        polygons[j] = polygon;
    }
    map.put(adcode, new DistrictPolyline(adcode, polygons));
});
polylinesInStr.clear();
CITY_POLYLINE_MAP = ImmutableMap.copyOf(map);

根据GPS遍历多边形MAP,获取adcode

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
public static Optional<String> getAdCode(double lon, double lat) {
    Coordinate coordinate = new Coordinate(lon, lat);
    Iterator<Map.Entry<String, DistrictPolyline>> iterator = CITY_POLYLINE_MAP.entrySet().iterator();
    while (iterator.hasNext()) {
        Map.Entry<String, DistrictPolyline> next = iterator.next();
        DistrictPolyline value = next.getValue();
        Polygon[] polygons = value.polygons;
        for (Polygon polygon : polygons) {
            // 调用JTS库,检查坐标点是否在polygon中
            boolean contains = SimplePointInAreaLocator.containsPointInPolygon(coordinate, polygon);
            if (contains) {
                // 访问计数,用于检索速度
                value.count++;
                return Optional.of(next.getKey());
            }
        }
    }
    return Optional.empty();
}

优化 每次访问后访问计数+1,定时根据访问计数重置区划在map中的位置,让最多访问的行政区划最先遍历到

1
2
3
4
5
6
7
8
9
List<DistrictPolyline> polylines = Lists.newLinkedList(CITY_POLYLINE_MAP.values());
Collections.sort(polylines, (o1, o2) -> Long.compare(o2.count, o1.count));
Map<String, DistrictPolyline> map = Maps.newLinkedHashMap();
int size = polylines.size();
for (DistrictPolyline p : polylines) {
    p.count = size--;
    map.put(p.adcode, p);
}
CITY_POLYLINE_MAP = ImmutableMap.copyOf(map);
音视频

turnserver配置

下面是找到的一个唯一可用的turnserver的配置,网上大部分是无法使用的

2 部署问题

2.1 turn网络报错errno=99

2.1.1 描述

1
bind: Cannot assign requested address 209: Trying to bind fd 429 to <xx.xx.xx.xx:58896>: errno=99

2.1.2 解决方法

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# 修改配置文件:/etc/network/interfaces.d/50-cloud-init.cfg 添加 eth0:0 的外网IP配置
 
auto eth0:0
iface eth0:0 inet static
    address <外网IP>
    netmask 255.255.255.0
    post-up route add default gw xx.xx.xx.xx || true
    pre-down route del default gw xx.xx.xx.xx || true
 
 
# 重启网络 /etc/init.d/networking restart
 
ifconfig
 
 
...
eth0:0    Link encap:Ethernet  HWaddr xx:xx:xx:xx:xx:xx
          inet addr:{IP1}  Bcast:152.136.63.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

附录:完整的配置文件

turnserver.config

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
# they do not support STUN RFC 5780 functionality (CHANGE REQUEST).
#
# 2) Auxiliary servers also are never returning ALTERNATIVE-SERVER reply.
# 
# Valid formats are 1.2.3.4:5555 for IPv4 and [1:2::3:4]:5555 for IPv6.
#
# There may be multiple aux-server options, each will be used for listening
# to client requests.
#
#aux-server=101.200.12.36:3478
#aux-server=[2607:f0d0:1002:51::4]:33478
 
# (recommended for older Linuxes only)
# Automatically balance UDP traffic over auxiliary servers (if configured).
# The load balancing is using the ALTERNATE-SERVER mechanism.
# The TURN client must support 300 ALTERNATE-SERVER response for this 
# functionality.
#
#udp-self-balance
 
# Relay interface device for relay sockets (optional, Linux only).
# NOT RECOMMENDED.
#
#relay-device=eth1
 
# Relay address (the local IP address that will be used to relay the 
# packets to the peer).
# Multiple relay addresses may be used.
# The same IP(s) can be used as both listening IP(s) and relay IP(s).
#
# If no relay IP(s) specified, then the turnserver will apply the default
# policy: it will decide itself which relay addresses to be used, and it 
# will always be using the client socket IP address as the relay IP address
# of the TURN session (if the requested relay address family is the same
# as the family of the client socket).
#
#relay-ip=101.200.12.36
relay-ip=<外网IP>
#relay-ip=2607:f0d0:1002:51::5
 
# For Amazon EC2 users:
#
# TURN Server public/private address mapping, if the server is behind NAT.
# In that situation, if a -X is used in form "-X <ip>" then that ip will be reported
# as relay IP address of all allocations. This scenario works only in a simple case
# when one single relay address is be used, and no RFC5780 functionality is required.
# That single relay address must be mapped by NAT to the 'external' IP.
# The "external-ip" value, if not empty, is returned in XOR-RELAYED-ADDRESS field.
# For that 'external' IP, NAT must forward ports directly (relayed port 12345
# must be always mapped to the same 'external' port 12345).
#
# In more complex case when more than one IP address is involved,
# that option must be used several times, each entry must
# have form "-X <public-ip/private-ip>", to map all involved addresses.
# RFC5780 NAT discovery STUN functionality will work correctly,
# if the addresses are mapped properly, even when the TURN server itself 
# is behind A NAT.
#
# By default, this value is empty, and no address mapping is used.
#
external-ip=<外网IP>
#
#OR:
#
#external-ip=60.70.80.91/172.17.19.101
#external-ip=60.70.80.92/172.17.19.102
 
 
# Number of relay threads to handle the established connections
# (in addition to authentication thread and the listener thread).
# If set to 0 then application runs relay process in a single thread,
# in the same thread with the listener process (the authentication thread will 
# still be a separate thread).
#
# In the older systems (Linux kernel before 3.9),
# the number of UDP threads is always one thread per network listening endpoint - 
# including the auxiliary endpoints - unless 0 (zero) or 1 (one) value is set.
#
#relay-threads=0
 
# Lower and upper bounds of the UDP relay endpoints:
# (default values are 49152 and 65535)
#
#min-port=49152
#max-port=65535
 
# Uncomment to run TURN server in 'normal' 'moderate' verbose mode.
# By default the verbose mode is off.
verbose
 
# Uncomment to run TURN server in 'extra' verbose mode.
# This mode is very annoying and produces lots of output.
# Not recommended under any normal circumstances.
#
#Verbose
 
# Uncomment to use fingerprints in the TURN messages.
# By default the fingerprints are off.
#
#fingerprint
 
# Uncomment to use long-term credential mechanism.
# By default no credentials mechanism is used (any user allowed).
# This option can be used with either flat file user database or 
# PostgreSQL DB or MySQL DB or Redis DB for user keys storage.
#
lt-cred-mech
 
# Uncomment to use short-term credential mechanism.
# By default no credentials mechanism is used (any user allowed).
# For short-term credential mechanism you have to use PostgreSQL or 
# MySQL or Redis database for user password storage.
#
#st-cred-mech
 
# This option is opposite to lt-cred-mech or st-cred-mech. 
# (TURN Server with no-auth option allows anonymous access).
# If neither option is defined, and no users are defined,
# then no-auth is default. If at least one user is defined, 
# in this file or in command line or in usersdb file, then
# lt-cred-mech is default.
#
#no-auth
 
# TURN REST API flag.
# Flag that sets a special authorization option that is based upon authentication secret.
# This feature can be used with the long-term authentication mechanism, only.
# This feature purpose is to support "TURN Server REST API", see
# "TURN REST API" link in the project's page 
# http://code.google.com/p/rfc5766-turn-server/.
#
# This option is used with timestamp:
# 
# usercombo -> "timestamp:userid"
# turn user -> usercombo
# turn password -> base64(hmac(secret key, usercombo))
#
# This allows TURN credentials to be accounted for a specific user id.
# If you don't have a suitable id, the timestamp alone can be used.
# This option is just turning on secret-based authentication.
# The actual value of the secret is defined either by option static-auth-secret,
# or can be found in the turn_secret table in the database (see below).
# 
#use-auth-secret
 
# 'Static' authentication secret value (a string) for TURN REST API only. 
# If not set, then the turn server
# will try to use the 'dynamic' value in turn_secret table
# in user database (if present). The database-stored  value can be changed on-the-fly
# by a separate program, so this is why that other mode is 'dynamic'.
#
#static-auth-secret 
 
# 'Static' user accounts for long term credentials mechanism, only.
# This option cannot be used with TURN REST API or with short-term credentials
# mechanism.
# 'Static' user accounts are NOT dynamically checked by the turnserver process, 
# so that they can NOT be changed while the turnserver is running.
#
#user=username1:key1
#user=username2:key2
# OR:
#user=username1:password1
#user=username2:password2
 
user=<用户名>:<用户的Key或密码也可以使用DB动态的配置>
#
#account_method="file"
#account_file="/home/debug/coturn/turnusers.txt"
 
# Keys must be generated by turnadmin utility. The key value depends
# on user name, realm, and password:
#
# Example:
# $ turnadmin -k -u ninefingers -r north.gov -p youhavetoberealistic
# Output: 0xbc807ee29df3c9ffa736523fb2c4e8ee
# ('0x' in the beginning of the key is what differentiates the key from
# password. If it has 0x then it is a key, otherwise it is a password).
#
# The corresponding user account entry in the config file will be:
# 
#user=ninefingers:0xbc807ee29df3c9ffa736523fb2c4e8ee
# Or, equivalently, with open clear password (less secure):
#user=ninefingers:youhavetoberealistic
#
 
# 'Dynamic' user accounts database file name.
# Only users for long-term mechanism can be stored in a flat file,
# short-term mechanism will not work with option, the short-term
# mechanism required PostgreSQL or MySQL or Redis database.
# 'Dynamic' long-term user accounts are dynamically checked by the turnserver process, 
# so that they can be changed while the turnserver is running.
#
# Default file name is turnuserdb.conf.
# 
#userdb=/usr/local/etc/turnuserdb.conf
 
# PostgreSQL database connection string in the case that we are using PostgreSQL
# as the user database.
# This database can be used for long-term and short-term credential mechanisms
# and it can store the secret value for secret-based timed authentication in TURN RESP API. 
# See http://www.postgresql.org/docs/8.4/static/libpq-connect.html for 8.x PostgreSQL
# versions connection string format, see 
# http://www.postgresql.org/docs/9.2/static/libpq-connect.html#LIBPQ-CONNSTRING
# for 9.x and newer connection string formats.
#
#psql-userdb="host=<host> dbname=<database-name> user=<database-user> password=<database-user-password> connect_timeout=30"
 
# MySQL database connection string in the case that we are using MySQL
# as the user database.
# This database can be used for long-term and short-term credential mechanisms
# and it can store the secret value for secret-based timed authentication in TURN RESP API. 
# Use string format as below (space separated parameters, all optional):
#
#mysql-userdb="host=<host> dbname=<database-name> user=<database-user> password=<database-user-password> port=<port> connect_timeout=<seconds>"
# mysql-userdb="host=localhost dbname=coturn user=root password=mysql1 port=3306 connect_timeout=30"
 
 
# Redis database connection string in the case that we are using Redis
# as the user database.
# This database can be used for long-term and short-term credential mechanisms
# and it can store the secret value for secret-based timed authentication in TURN RESP API. 
# Use string format as below (space separated parameters, all optional):
#
redis-userdb="ip=<ip-address> dbname=<database-number> password=<database-user-password> port=<port> connect_timeout=<seconds可以设置为30>"
 
# Redis status and statistics database connection string, if used (default - empty, no Redis stats DB used).
# This database keeps allocations status information, and it can be also used for publishing
# and delivering traffic and allocation event notifications.
# The connection string has the same parameters as redis-userdb connection string. 
# Use string format as below (space separated parameters, all optional):
#
#redis-statsdb="ip=<ip-address> dbname=<database-number> password=<database-user-password> port=<port> connect_timeout=<seconds>"
 
# Realm for long-term credentials mechanism and for TURN REST API.
#
#realm=coturn.com
realm=<如果是使用key的方式配置用户此处就必须配置生成Key时使用参数>
 
# Per-user allocation quota.
# default value is 0 (no quota, unlimited number of sessions per user).
#
#user-quota=0
 
# Total allocation quota.
# default value is 0 (no quota).
#
#total-quota=0
 
# Max bytes-per-second bandwidth a TURN session is allowed to handle
# (input and output network streams are treated separately). Anything above
# that limit will be dropped or temporary suppressed (within
# the available buffer limits).
#
#max-bps=0
 
# Uncomment if no UDP client listener is desired.
# By default UDP client listener is always started.
#
#no-udp
 
# Uncomment if no TCP client listener is desired.
# By default TCP client listener is always started.
#
#no-tcp
 
# Uncomment if no TLS client listener is desired.
# By default TLS client listener is always started.
#
#no-tls
 
# Uncomment if no DTLS client listener is desired.
# By default DTLS client listener is always started.
#
#no-dtls
 
# Uncomment if no UDP relay endpoints are allowed.
# By default UDP relay endpoints are enabled (like in RFC 5766).
#
#no-udp-relay
 
# Uncomment if no TCP relay endpoints are allowed.
# By default TCP relay endpoints are enabled (like in RFC 6062).
#
#no-tcp-relay
 
# Uncomment if extra security is desired,
# with nonce value having limited lifetime (600 secs).
# By default, the nonce value is unique for a session,
# but it has unlimited lifetime. With this option,
# the nonce lifetime is limited to 600 seconds, after that 
# the client will get 438 error and will have to re-authenticate itself.
# 修改回话永不超时
stale-nonce
 
# Certificate file.
# Use an absolute path or path relative to the 
# configuration file.
#
#cert=/usr/local/etc/turn_server_cert.pem
 
# Private key file.
# Use an absolute path or path relative to the 
# configuration file.
# Use PEM file format.
#
#pkey=/usr/local/etc/turn_server_pkey.pem
 
# Private key file password, if it is in encoded format.
# This option has no default value.
#
#pkey-pwd=...
 
# Allowed OpenSSL cipher list for TLS/DTLS connections.
# Default value is "DEFAULT".
#
#cipher-list="DEFAULT"
 
# CA file in OpenSSL format. 
# Forces TURN server to verify the client SSL certificates.
# By default it is not set: there is no default value and the client
# certificate is not checked.
#
# Example:
#CA-file=/etc/ssh/id_rsa.cert
 
# Curve name for EC ciphers, if supported by OpenSSL library (TLS and DTLS).
# The default value is prime256v1.
#
#ec-curve-name=prime256v1
 
# Use 566 bits predefined DH TLS key. Default size of the key is 1066.
#
#dh566
 
# Use 2066 bits predefined DH TLS key. Default size of the key is 1066.
#
#dh2066
 
# Use custom DH TLS key, stored in PEM format in the file.
# Flags --dh566 and --dh2066 are ignored when the DH key is taken from a file.
#
#dh-file=<DH-PEM-file-name>
 
# Flag to prevent stdout log messages.
# By default, all log messages are going to both stdout and to 
# the configured log file. With this option everything will be 
# going to the configured log only (unless the log file itself is stdout).
#
#no-stdout-log
 
# Option to set the log file name.
# By default, the turnserver tries to open a log file in 
# /var/log, /var/tmp, /tmp and current directories directories
# (which open operation succeeds first that file will be used).
# With this option you can set the definite log file name.
# The special names are "stdout" and "-" - they will force everything 
# to the stdout. Also, the "syslog" name will force everything to
# the system log (syslog). 
# In the runtime, the logfile can be reset with the SIGHUP signal 
# to the turnserver process.
#
log-file=/var/tmp/turn.log
 
# Option to redirect all log output into system log (syslog).
#
#syslog
 
# This flag means that no log file rollover will be used, and the log file
# name will be constructed as-is, without PID and date appendage.
#simple-log
 
# Option to set the "redirection" mode. The value of this option
# will be the address of the alternate server for UDP & TCP service in form of 
# <ip>[:<port>]. The server will send this value in the attribute
# ALTERNATE-SERVER, with error 300, on ALLOCATE request, to the client.
# Client will receive only values with the same address family
# as the client network endpoint address family. 
# See RFC 5389 and RFC 5766 for ALTERNATE-SERVER functionality description. 
# The client must use the obtained value for subsequent TURN communications.
# If more than one --alternate-server options are provided, then the functionality
# can be more accurately described as "load-balancing" than a mere "redirection". 
# If the port number is omitted, then the default port 
# number 3478 for the UDP/TCP protocols will be used.
# Colon (:) characters in IPv6 addresses may conflict with the syntax of 
# the option. To alleviate this conflict, literal IPv6 addresses are enclosed 
# in square brackets in such resource identifiers, for example: 
# [2001:db8:85a3:8d3:1319:8a2e:370:7348]:3478 . 
# Multiple alternate servers can be set. They will be used in the
# round-robin manner. All servers in the pool are considered of equal weight and 
# the load will be distributed equally. For example, if we have 4 alternate servers, 
# then each server will receive 25% of ALLOCATE requests. A alternate TURN server 
# address can be used more than one time with the alternate-server option, so this 
# can emulate "weighting" of the servers.
#
# Examples: 
#alternate-server=1.2.3.4:5678
#alternate-server=11.22.33.44:56789
#alternate-server=5.6.7.8
#alternate-server=[2001:db8:85a3:8d3:1319:8a2e:370:7348]:3478
 
# Option to set alternative server for TLS & DTLS services in form of 
# <ip>:<port>. If the port number is omitted, then the default port 
# number 5349 for the TLS/DTLS protocols will be used. See the previous 
# option for the functionality description.
#
# Examples: 
#tls-alternate-server=1.2.3.4:5678
#tls-alternate-server=11.22.33.44:56789
#tls-alternate-server=[2001:db8:85a3:8d3:1319:8a2e:370:7348]:3478
 
# Option to suppress TURN functionality, only STUN requests will be processed.
# Run as STUN server only, all TURN requests will be ignored.
# By default, this option is NOT set.
#
#stun-only
 
# Option to suppress STUN functionality, only TURN requests will be processed.
# Run as TURN server only, all STUN requests will be ignored.
# By default, this option is NOT set.
#
#no-stun
 
# This is the timestamp/username separator symbol (character) in TURN REST API.
# The default value is ':'.
# rest-api-separator=:
 
# Flag that can be used to disallow peers on the loopback addresses (127.x.x.x and ::1).
# This is an extra security measure.
#
#no-loopback-peers
 
# Flag that can be used to disallow peers on well-known broadcast addresses (224.0.0.0 and above, and FFXX:*).
# This is an extra security measure.
#
#no-multicast-peers
 
# Option to set the max time, in seconds, allowed for full allocation establishment. 
# Default is 60 seconds.
#
#max-allocate-timeout=60
 
# Option to allow or ban specific ip addresses or ranges of ip addresses. 
# If an ip address is specified as both allowed and denied, then the ip address is 
# considered to be allowed. This is useful when you wish to ban a range of ip 
# addresses, except for a few specific ips within that range.
#
# This can be used when you do not want users of the turn server to be able to access
# machines reachable by the turn server, but would otherwise be unreachable from the 
# internet (e.g. when the turn server is sitting behind a NAT)
#
# Examples:
# denied-peer-ip=83.166.64.0-83.166.95.255
# allowed-peer-ip=83.166.68.45
 
# File name to store the pid of the process.
# Default is /var/run/turnserver.pid (if superuser account is used) or
# /var/tmp/turnserver.pid .
#
#pidfile="/var/run/turnserver.pid"
 
# Require authentication of the STUN Binding request.
# By default, the clients are allowed anonymous access to the STUN Binding functionality.
#
#secure-stun
 
# Require SHA256 digest function to be used for the message integrity.
# By default, the server uses SHA1 (as per TURN standard specs). 
# With this option, the server 
# always requires the stronger SHA256 function. The client application
# must support SHA256 hash function if this option is used. If the server obtains 
# a message from the client with a weaker (SHA1) hash function then the 
# server returns error code 426.
#
#sha256
 
# Mobility with ICE (MICE) specs support.
#
#mobility
 
# User name to run the process. After the initialization, the turnserver process
# will make an attempt to change the current user ID to that user.
#
#proc-user=<user-name>
 
# Group name to run the process. After the initialization, the turnserver process
# will make an attempt to change the current group ID to that group.
#
#proc-group=<group-name>
 
# Turn OFF the CLI support.
# By default it is always ON.
# See also options cli-ip and cli-port.
#
#no-cli
 
#Local system IP address to be used for CLI server endpoint. Default value
# is 127.0.0.1.
#
#cli-ip=127.0.0.1
 
# CLI server port. Default is 5766.
#
#cli-port=5766
 
# CLI access password. Default is empty (no password).
#
#cli-password=logen
 
# Server relay. NON-STANDARD AND DANGEROUS OPTION. 
# Only for those applications when we want to run 
# server applications on the relay endpoints.
# This option eliminates the IP permissions check on 
# the packets incoming to the relay endpoints.
#
#server-relay
 
# Maximum number of output sessions in ps CLI command.
# This value can be changed on-the-fly in CLI. The default value is 256.
#
#cli-max-output-sessions
 
# Set network engine type for the process (for internal purposes).
#
#ne=[1|2|3]
 
# Do not allow an SSL/TLS version of protocol
#
#no-sslv2
#no-sslv3
#no-tlsv1
#no-tlsv1_1
#no-tlsv1_2
音视频

基于Gstreamer的rtp转rtmp代码

rtp2rtmp_video.c
========

1. 推流到rtmp服务器

1
2
3
 
$ ffmpeg -re -i ./fulankelin-hd.mp4 -an -vcodec h264 -f rtp rtp://127.0.0.1:5004 -vn -acodec libopus -f rtp rtp://127.0.0.1:5003
$

2. SDP信息

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
SDP:
 
SDP:
v=0
o=- 0 0 IN IP4 127.0.0.1
s=No Name
t=0 0
a=tool:libavformat 58.29.100
m=video 5004 RTP/AVP 96
c=IN IP4 127.0.0.1
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1
m=audio 5003 RTP/AVP 97
c=IN IP4 127.0.0.1
b=AS:96
a=rtpmap:97 opus/48000/2
a=fmtp:97 sprop-stereo=1
 
.

3. gst-launch测试

1
2
3
4
5
6
7
8
9
10
 
gst-launch-1.0 -em \
  rtpbin name=rtpbin latency=5 \
  udpsrc port=5003 caps="application/x-rtp,media=(string)audio,clock-rate=(int)48000,encoding-name=(string)OPUS" ! rtpbin.recv_rtp_sink_0 \
    rtpbin.  ! rtpopusdepay ! opusdec ! audioconvert ! audioresample ! avenc_aac ! mux. \
  udpsrc port=5004 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264" ! rtpbin.recv_rtp_sink_1 \
    rtpbin.  ! rtph264depay ! h264parse ! mux. \
  flvmux name=mux streamable=true ! rtmpsink sync=false location=rtmp://u1802/live/demo
 
.

4. 程序代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
 
#include <string.h>
#include <math.h>
 
#include <gst/gst.h>
 
#define VIDEO_CAPS "application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264"
#define AUDIO_CAPS "application/x-rtp,media=(string)audio,clock-rate=(int)48000,encoding-name=(string)OPUS"
 
/* will be called when rtpbin has validated a payload that we can depayload */
static void
pad_added_cb(GstElement *rtpbin, GstPad *new_pad, GstElement *depay)
{
    char *pad_name = GST_PAD_NAME(new_pad);
    char *depay_name = gst_element_get_name(depay);
    if (strstr(pad_name, "recv_rtp_src_0_") && strstr(depay_name, "audiodepay"))
    {
        GstPad *sinkpad;
        GstPadLinkReturn lres;
 
        g_print("new payload on rtpbin: %s %s %s\n",
                gst_element_get_name(rtpbin), GST_PAD_NAME(new_pad), gst_element_get_name(depay));
 
        sinkpad = gst_element_get_static_pad(depay, "sink");
        g_assert(sinkpad);
 
        lres = gst_pad_link(new_pad, sinkpad);
        g_assert(lres == GST_PAD_LINK_OK);
        gst_object_unref(sinkpad);
    }
    else if (strstr(pad_name, "recv_rtp_src_1_") && strstr(depay_name, "videodepay"))
    {
        GstPad *sinkpad;
        GstPadLinkReturn lres;
 
        g_print("new payload on rtpbin: %s %s %s\n",
                gst_element_get_name(rtpbin), GST_PAD_NAME(new_pad), gst_element_get_name(depay));
 
        sinkpad = gst_element_get_static_pad(depay, "sink");
        g_assert(sinkpad);
 
        lres = gst_pad_link(new_pad, sinkpad);
        g_assert(lres == GST_PAD_LINK_OK);
        gst_object_unref(sinkpad);
    }
}
 
int main(int argc, char *argv[])
{
    GMainLoop *loop;
    GstElement *pipeline;
 
    GstElement *rtpbin;
    GstElement *audiosrc, *audiodepay, *audiodec, *audiores, *audioconv, *audiosink;
    GstElement *videosrc, *videodepay, *videosink;
    GstElement *flvmux, *rtmpsink;
 
    gboolean res;
    GstCaps *caps;
    GstPadLinkReturn lres;
    GstPad *srcpad, *audio_sinkpad, *video_sinkpad;
 
    gst_init(&argc, &argv);
    pipeline = gst_pipeline_new(NULL);
    g_assert(pipeline);
 
    /* the rtpbin element */
    rtpbin = gst_element_factory_make("rtpbin", "rtpbin");
    g_assert(rtpbin);
    gst_bin_add(GST_BIN(pipeline), rtpbin);
    // 001 源
    audiosrc = gst_element_factory_make("udpsrc", "audiosrc");
    g_assert(audiosrc);
    g_object_set(audiosrc, "port", 5003, NULL);
    caps = gst_caps_from_string(AUDIO_CAPS);
    g_object_set(audiosrc, "caps", caps, NULL);
    gst_caps_unref(caps);
    gst_bin_add(GST_BIN(pipeline), audiosrc);
 
    videosrc = gst_element_factory_make("udpsrc", "videosrc");
    g_assert(videosrc);
    g_object_set(videosrc, "port", 5004, NULL);
    caps = gst_caps_from_string(VIDEO_CAPS);
    g_object_set(videosrc, "caps", caps, NULL);
    gst_caps_unref(caps);
    gst_bin_add(GST_BIN(pipeline), videosrc);
 
    /* now link all to the rtpbin, start by getting an RTP sinkpad for session 0 */
    srcpad = gst_element_get_static_pad(audiosrc, "src");
    audio_sinkpad = gst_element_get_request_pad(rtpbin, "recv_rtp_sink_0");
    lres = gst_pad_link(srcpad, audio_sinkpad);
    g_assert(lres == GST_PAD_LINK_OK);
    gst_object_unref(srcpad);
 
    srcpad = gst_element_get_static_pad(videosrc, "src");
    video_sinkpad = gst_element_get_request_pad(rtpbin, "recv_rtp_sink_1");
    lres = gst_pad_link(srcpad, video_sinkpad);
    g_assert(lres == GST_PAD_LINK_OK);
    gst_object_unref(srcpad);
 
    /* the depayloading and decoding */
    audiodepay = gst_element_factory_make("rtpopusdepay", "audiodepay");
    g_assert(audiodepay);
    audiodec = gst_element_factory_make("opusdec", "audiodec");
    g_assert(audiodepay);
    /* the audio playback and format conversion */
    audioconv = gst_element_factory_make("audioconvert", "audioconv");
    g_assert(audioconv);
    audiores = gst_element_factory_make("audioresample", "audiores");
    g_assert(audiores);
    audiosink = gst_element_factory_make("avenc_aac", "audiosink"); // autoaudiosink voaacenc avenc_aac avenc_opus
    g_assert(audiosink);
    /* add depayloading and playback to the pipeline and link */
    gst_bin_add_many(GST_BIN(pipeline), audiodepay, audiodec, audioconv,
                     audiores, audiosink, NULL);
    res = gst_element_link_many(audiodepay, audiodec, audioconv, audiores,
                                audiosink, NULL);
    g_assert(res == TRUE);
 
    videodepay = gst_element_factory_make("rtph264depay", "videodepay");
    g_assert(videodepay);
    videosink = gst_element_factory_make("h264parse", "videosink");
    g_assert(videosink);
    gst_bin_add_many(GST_BIN(pipeline), videodepay, videosink, NULL);
    res = gst_element_link_many(videodepay, videosink, NULL);
    g_assert(res == TRUE);
 
    // flvmux
    flvmux = gst_element_factory_make("flvmux", "flvmux");
    g_assert(flvmux);
    g_object_set(flvmux, "streamable", TRUE, NULL);
    gst_bin_add(GST_BIN(pipeline), flvmux);
 
    res = gst_element_link(audiosink, flvmux);
    g_assert(res == TRUE);
    res = gst_element_link(videosink, flvmux);
    g_assert(res == TRUE);
 
    rtmpsink = gst_element_factory_make("rtmpsink", "rtmpsink");
    g_assert(rtmpsink);
    g_object_set(rtmpsink, "sync", FALSE, NULL);
    g_object_set(rtmpsink, "location", "rtmp://u1802/live/demo2", NULL);
    gst_bin_add(GST_BIN(pipeline), rtmpsink);
    res = gst_element_link(flvmux, rtmpsink);
    g_assert(res == TRUE);
 
    /* the RTP pad that we have to connect to the depayloader will be created
   * dynamically so we connect to the pad-added signal, pass the depayloader as
   * user_data so that we can link to it. */
    g_signal_connect(rtpbin, "pad-added", G_CALLBACK(pad_added_cb), audiodepay);
    g_signal_connect(rtpbin, "pad-added", G_CALLBACK(pad_added_cb), videodepay);
 
    /* set the pipeline to playing */
    g_print("starting receiver pipeline\n");
    gst_element_set_state(pipeline, GST_STATE_PLAYING);
 
    /* we need to run a GLib main loop to get the messages */
    loop = g_main_loop_new(NULL, FALSE);
    g_main_loop_run(loop);
 
    g_print("stopping receiver pipeline\n");
    gst_element_set_state(pipeline, GST_STATE_NULL);
 
    gst_object_unref(loop);
    gst_object_unref(pipeline);
    gst_object_unref(audio_sinkpad);
    gst_object_unref(video_sinkpad);
    gst_object_unref(rtmpsink);
    gst_object_unref(flvmux);
    gst_object_unref(rtpbin);
    gst_object_unref(audiosrc);
    gst_object_unref(audiodepay);
    gst_object_unref(audiodec);
    gst_object_unref(audiores);
    gst_object_unref(audioconv);
    gst_object_unref(audiosink);
    gst_object_unref(videosrc);
    gst_object_unref(videodepay);
    gst_object_unref(videosink);
    return 0;
}
 
//
ElaticSearch

elasticsearch增加磁盘

基于EasticSearch-6.2.4

检查系统状态

1
2
3
curl -XGET "http://192.168.1.1:9002/_cat/indices"
curl -XGET "http://192.168.1.1:9002/_cat/nodes"
curl -XGET "http://192.168.1.1:9002/_cat/health"

修改配置文件,增加数据目录,多个以逗号分隔

1
2
# vim config/elasticsearch.yml
path.data: /data2/es/data,/data1/es/data

检查系统状态

1
2
3
curl -XGET "http://192.168.1.1:9002/_cat/indices"
curl -XGET "http://192.168.1.1:9002/_cat/nodes"
curl -XGET "http://192.168.1.1:9002/_cat/health"

为避免节点重启造成分片重新自动分配,需要设置集群中分片不自动分配

1
2
3
4
5
6
7
8
9
curl -X PUT \
  http://192.168.1.1:9002/_cluster/settings \
  -H 'Content-Type: application/json; charset=UTF-8' \
  -H 'cache-control: no-cache' \
  -d '{
  "transient" : {
      "cluster.routing.allocation.enable" : "none"
  }
}'

关闭节点

1
2
jps -l
kill xxxx

启动服务

1
./bin/elasticsearch -d

开启分片自动分配

1
2
3
4
5
6
7
8
9
curl -X PUT \
  http://192.168.1.1:9002/_cluster/settings \
  -H 'Content-Type: application/json; charset=UTF-8' \
  -H 'cache-control: no-cache' \
  -d '{
  "transient": {
    "cluster.routing.allocation.enable": "all"
  }
}'

查看分片及索引状态

1
2
3
4
5
# 查看未分配分片数
curl -XGET 'http://192.168.1.1:9002/_cat/shards' | grep UNASSIGNED   #查看未分配的索引分片
curl -XGET "http://192.168.1.1:9002/_cat/shards/lbs_geocoder_4307?v" #查看索引分片
curl -XGET "http://192.168.1.1:9002/_cat/shards?v"
curl -XGET "http://192.168.1.1:9002/_cat/shards?v" |grep UNASSIGNED |wc -l

当没有未分配分片时,检查系统状态

1
2
3
curl -XGET "http://192.168.1.1:9002/_cat/indices"
curl -XGET "http://192.168.1.1:9002/_cat/nodes"
curl -XGET "http://192.168.1.1:9002/_cat/health"

因为负载过高等原因,有时候个别分片可能长期处于 UNASSIGNED 状态时,可以手动进行分配;reroute 接口支持五种指令:allocate_replica, allocate_stale_primary, allocate_empty_primary,move 和 cancel;默认情况下只允许手动分配副本分片(即使用 allocate_replica),所以如果要分配主分片,需要单独加一个 accept_data_loss 选项

分配主分片

1
2
3
4
5
6
7
8
9
10
11
12
13
curl -X PUT \
  http://192.168.1.1:9002/_cluster/reroute \
  -H 'Content-Type: application/json; charset=UTF-8' \
  -H 'cache-control: no-cache' \
  -d '{
  "commands" : [ {
        "allocate_stale_primary" :
            {
              "index" : "index", "shard" : 4, "node" : "node56", "accept_data_loss" : true
            }
        }
  ]
}'

分配副本

1
2
3
4
5
6
7
8
9
10
11
12
13
curl -X PUT \
  http://192.168.1.1:9002/_cluster/reroute \
  -H 'Content-Type: application/json; charset=UTF-8' \
  -H 'cache-control: no-cache' \
  -d '{
  "commands" : [ {
        "allocate_replica" :
            {
              "index" : "index", "shard" : 4, "node" : "node56"
            }
        }
  ]
}'

磁盘空间不足报 [FORBIDDEN/12/index read-only / allow delete (api)] 错误后,进行增加磁盘操作或删除无用旧索引数据,完成操作需要修改索引的状态,ES不会主动更新,命令为:

1
PUT _settings { "index":{ "blocks":{ "read_only_allow_delete":"false" } } }

磁盘扩容后,可能需要修改报警的设置,默认值会在 85% 90% 95%的磁盘空间时报警

1
2
3
4
5
6
7
8
9
10
PUT /_cluster/settings
{
  "transient": {
    "cluster.routing.allocation.disk.threshold_enabled" : true,
    "cluster.routing.allocation.disk.watermark.low": "80gb",
    "cluster.routing.allocation.disk.watermark.high": "40gb",
    "cluster.routing.allocation.disk.watermark.flood_stage":"20gb",
    "cluster.info.update.interval" : "1m"
  }
}
未分类

qcow2镜像定制指南

背景

目前网络上关于定制镜像的说明很分散,需要搜寻很多文章才能完成镜像的定制任务。所以我尝试提供一个全面而系统的指南,遵循本指南,用户可以方便的完成镜像的定制。

实施步骤

一、环境配置

1、准备软件
mac pro、VmWare fusion、CentOS-7-x86_64-DVD-1708.iso、CentOS-7-x86_64-GenericCloud-1708-20180123.qcow2
2、安装嵌套CentOs环境
由于MacOs不支持Kvm,故需要在嵌套的操作系统中安装云镜像需要的软件,使用Fusion很容易在MacOs中虚拟出一个CentOs的环境。
3、修改嵌套操作系统配置
在centos关闭的情况下打开虚拟机“处理器和内存”配置,选择“高级配置”,选中“在此虚拟机中启用虚拟化管理程序”和“在此虚拟机中启用代码分析应用程序”,如无这步操作,则在启动virt-manager时会报:“
virt-manager 报 WARNING : KVM不可用.这可能是因为没有安装KVM软件包,或者没有载入KVM内核模块.您的虚拟机可能性很差。”的错误,启动虚拟机。以下操作如无特殊说明都是在嵌套操作系统中执行。
4、安装依赖

1
2
3
yum install qemu-kvm qemu-img qemu-kvm-tools qemu-kvm-common
yum install libvirt-admin libvirt-client libvirt-daemon libvirt-devel libvirt
yum install libguestfs libguestfs-tools libguestfs-bash-completion

5、编译nbd内核模块(如不使用“nbd挂载方式修改镜像”则不需要安装此模块)
执行命令,出现以下报错时,说明没有nbd模块,需要自己手动安装

1
2
modprobe nbd
modprobe: FATAL: Module nbd not found.

执行下面的命令安装hbd模块

1
2
3
4
5
6
[@xx] # cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core)
[@xx] # uname -a
Linux localhost 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
[] # uname -r
3.10.0-693.el7.x86_64

安装kernel组件

1
[@xx] sudo yum install kernel-devel kernel-headers

编译安装hbd组件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# 下载对应的内核源码包
[@xx] # wget http://vault.centos.org/7.4.1708/os/Source/SPackages/kernel-3.10.0-693.el7.src.rpm
[@xx] # rpm -ihv kernel-3.10.0-693.el7.src.rpm
[@xx] # cd /root/rpmbuild/SOURCES/
[@xx] # tar Jxvf linux-3.10.0-123.el7.tar.xz -C /usr/src/kernels/
[@xx] # cd /usr/src/kernels/
# 配置源码
[@xx] # mv $(uname -r) $(uname -r)-old
[@xx] # mv linux-3.10.0-693.el7 $(uname -r)
# 编译安装
[@xx 3.10.0-693.el7.x86_64] # cd $(uname -r)
[@xx 3.10.0-693.el7.x86_64] # make mrproper
[@xx 3.10.0-693.el7.x86_64] # cp ../$(uname -r)-old/Module.symvers ./
[@xx 3.10.0-693.el7.x86_64] # cp /boot/config-$(uname -r) ./.config
[@xx 3.10.0-693.el7.x86_64] # make oldconfig
[@xx 3.10.0-693.el7.x86_64] # make prepare
[@xx 3.10.0-693.el7.x86_64] # make scripts
[@xx 3.10.0-693.el7.x86_64] # make CONFIG_BLK_DEV_NBD=m M=drivers/block
[@xx 3.10.0-693.el7.x86_64] # cp drivers/block/nbd.ko /lib/modules/$(uname -r)/kernel/drivers/block/
[@xx 3.10.0-693.el7.x86_64] # depmod -a
# 查看nbd模块
[@xx 3.10.0-693.el7.x86_64]$ modinfo nbd
filename:       /lib/modules/3.10.0-693.el7.x86_64/kernel/drivers/block/nbd.ko
license:        GPL
description:    Network Block Device
rhelversion:    7.4
srcversion:     EDE909A294AC5FE08E81957
depends:        
vermagic:       3.10.0 SMP mod_unload modversions 
parm:           nbds_max:number of network block devices to initialize (default: 16) (int)
parm:           max_part:number of partitions per device (default: 0) (int)
parm:           debugflags:flags for controlling debug output (int)

编译安装时的错误处理
阶段:make CONFIG_BLK_DEV_NBD=m M=drivers/block

1
2
3
4
5
6
7
drivers/block/nbd.c: 在函数‘__nbd_ioctl’中:
drivers/block/nbd.c:619:19: 错误:‘REQ_TYPE_SPECIAL’未声明(在此函数内第一次使用)
   sreq.cmd_type = REQ_TYPE_SPECIAL;
                   ^
drivers/block/nbd.c:619:19: 附注:每个未声明的标识符在其出现的函数内只报告一次
make[1]: *** [drivers/block/nbd.o] 错误 1
make: *** [_module_drivers/block] 错误 2

处理:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
[@xx 3.10.0-693.el7.x86_64] # vim include/linux/blkdev.h
# 由代码可知 REQ_TYPE_SPECIAL = 7
/*
 * request command types
 */
enum rq_cmd_type_bits {
        REQ_TYPE_FS             = 1,    /* fs request */
        REQ_TYPE_BLOCK_PC,              /* scsi command */
        REQ_TYPE_SENSE,                 /* sense request */
        REQ_TYPE_PM_SUSPEND,            /* suspend request */
        REQ_TYPE_PM_RESUME,             /* resume request */
        REQ_TYPE_PM_SHUTDOWN,           /* shutdown request */
#ifdef __GENKSYMS__
        REQ_TYPE_SPECIAL,               /* driver defined type */
#else
        REQ_TYPE_DRV_PRIV,              /* driver defined type */
#endif
        /*
         * for ATA/ATAPI devices. this really doesn"&gt;cmd[0] with the range of driver
         * private REQ_LB opcodes to differentiate what type of request this is
         */
        REQ_TYPE_ATA_TASKFILE,
        REQ_TYPE_ATA_PC,
};
# 修改nbd.c文件
[@xx 3.10.0-693.el7.x86_64] # vim drivers/block/nbd.c
# sreq.cmd_type = REQ_TYPE_SPECIAL;
sreq.cmd_type = 7;
# 重新执行命令
[@xx 3.10.0-693.el7.x86_64] # make CONFIG_BLK_DEV_NBD=m M=drivers/block

二、设置镜像共享

设置嵌套虚拟机文件夹共享
qcow2文件放置在mac本地文件夹中,嵌套虚拟机通过文件共享的方式使用qcow2文件。需要注意的是qcow2文件权限需要在macos中设置为可读写,否则在嵌套虚拟机中无法更新配置。

1
[mac@] # chmod 755 ./CentOS-7-x86_64-GenericCloud-1708-20180123.qcow2

嵌套虚拟机中,需要要关闭SeLinux否则同样无法更新镜像内容

1
sudo /usr/sbin/setenforce 0

三、guestfish工具使用

1、示例程序:获取镜像ip地址

1
2
3
4
5
6
guestfish --rw -a ./CentOS-7-x86_64-GenericCloud-1708-20180123.qcow2
# run
# list-filesystems
/dev/sda1: xfs
# mount /dev/sda1 /
# vi /var/log/cloud-init.log

2、示例程序:配置用户访问权限

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
guestfish --rw -a ./CentOS-7-x86_64-GenericCloud-1708-20180123.qcow2
# run
# list-filesystems
/dev/sda1: xfs
# mount /dev/sda1 /
# mkdir /home/dc2-user/.ssh
# 注意guestfish与shell的区别,权限位是四位
# chown 1001 1001 /home/dc2-user/.ssh
# chmod 0700 /home/dc2-user/.ssh
# touch /home/dc2-user/.ssh/authorized_keys
# chmod 0600 /home/dc2-user/.ssh/authorized_keys
# chown 1001 1001 /home/dc2-user/.ssh/authorized_keys
# ll /home/dc2-user/.ssh/authorized_keys
-rw------- 1 1001 1001 0 Mar  1 10:05 /sysroot/home/dc2-user/.ssh/authorized_keys
# vi /home/dc2-user/.ssh/authorized_keys
# 添加公钥...
# quit

四、nbd挂载方式修改镜像(qemu-nbd)

1、确保已安装nbd模块,加载模块

1
2
# modinfo nbd
# insmod /lib/modules/3.10.0-693.el7.x86_64/kernel/drivers/block/nbd.ko max_part=8

3、建立nbd连接,挂载到目录

1
2
3
# qemu-nbd -c /dev/nbd0 ./CentOS-7-x86_64-GenericCloud-1708-20180123.qcow2
# mkdir -p /mnt/qcows/qcow0
# mount /dev/nbd0p1  /mnt/qcows/qcow0

4、执行chroot

1
# chroot /mnt/qcows/qcow0/

5、执行修改,比如

1
2
$ passwd dc2-user
# 或其它操作

6、修改完毕后解除挂载点,解除连接

1
2
# umount /mnt/qcows/qcow0
# qemu-nbd -d /dev/nbd0p1

五、通过virt-manager挂载虚拟机

1、执行

1
virt-manager

2、新建虚拟机
选择“导入现有磁盘”,“使用ISO镜像”,选择qcow2文件…
如果报:“WARNING : KVM不可用.这可能是因为没有安装KVM软件包,或者没有载入KVM内核模块.您的虚拟机可能性很差。”的警告相应的解决方案是:

  • 关闭虚拟机
  • 进入虚拟机设置(可以配置网卡,硬盘,光驱的地方)
  • 点击“处理器和内存”,勾选虚拟化Inter VT-x/EPT 或AMD-V/RVI(V)

3、登录机器,修改
IP地址和密码、公钥方式登录,在前面已说明如何操作

六、清理痕迹

1、清理/var/log/文件夹
2、删除cloud-init执行记录
cloud-init是专门为云环境的虚拟机初始化而开发的工具,通过读取相应的数据,对虚拟机进行配置。其执行完毕后会在一个叫 sem 的目录下创建信号文件,以便下次启动模块时不再重复执行语句。其目录位置为:
/var/lib/cloud/instances/实例ID/sem

1
2
sudo rm -rf /var/lib/cloud/instances
sudo rm -rf /var/lib/cloud/instance

3、清理history和.ssh目录等

1
2
3
4
5
rm -rf /home/xxx/.ssh
echo '' # /home/xxx/.bash_history
echo '' # /root/.bash_history
 
rm -rf /root/.oracle_jre_usage

七、去除磁盘空洞

1
2
3
4
5
6
# 创建同样大小的镜像
$ qemu-img create -f qcow2 CentOS-7-x86_64-GenericCloud-1708-20180329.qcow2 40G
$ virt-sparsify -x ./CentOS-7-x86_64-GenericCloud-1708-20180123.qcow2 --convert qcow2 ./CentOS-7-x86_64-GenericCloud-1708-20180329.qcow2
$ du -sh *
7.3G	CentOS-7-x86_64-GenericCloud-1708-20180123.qcow2
5.3G	CentOS-7-x86_64-GenericCloud-1708-20180329.qcow2