-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathatom.xml
More file actions
502 lines (281 loc) · 347 KB
/
Copy pathatom.xml
File metadata and controls
502 lines (281 loc) · 347 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>郑勇-个人博客</title>
<subtitle>每多学一点知识，就少写一行代码</subtitle>
<link href="/atom.xml" rel="self"/>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/"/>
<updated>2020-04-23T04:32:58.324Z</updated>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/</id>
<author>
<name>郑勇</name>
</author>
<generator uri="http://hexo.io/">Hexo</generator>
<entry>
<title>Livy部署及使用文档</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2020/04/23/Livy%E9%83%A8%E7%BD%B2%E5%8F%8A%E4%BD%BF%E7%94%A8%E6%96%87%E6%A1%A3/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2020/04/23/Livy部署及使用文档/</id>
<published>2020-04-23T03:04:33.000Z</published>
<updated>2020-04-23T04:32:58.324Z</updated>
<content type="html"><![CDATA[<h3 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h3><pre><code>本次部署是在单台机器10.57.17.215上部署，只部署了spark和livy；通过配置livy加载本地文件jar；然后通过livy rest api提交spark任务；</code></pre><h4 id="一、前置条件"><a href="#一、前置条件" class="headerlink" title="一、前置条件"></a>一、前置条件</h4><pre><code>1.安装spark: (部署参考：https://zyongjava.github.io/2019/12/24/Spark%E9%9B%86%E7%BE%A4%E9%83%A8%E7%BD%B2/)2.安装hadoop(可选，没有使用本地文件测试)</code></pre><h4 id="二、安装部署"><a href="#二、安装部署" class="headerlink" title="二、安装部署"></a>二、安装部署</h4><p>1）下载安装包</p><pre><code>http://livy.incubator.apache.org/download/</code></pre><p>2） 创建配置文件<code>livy.conf</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">cd apache-livy-0.7.0-incubating-bin/conf</span><br><span class="line">cp livy.conf.template livy.conf</span><br><span class="line">vi livy.conf</span><br></pre></td></tr></table></figure><p><code>livy.conf</code>中修改内容如下：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"># What host address to start the server on. By default, Livy will bind to all network interfaces.</span><br><span class="line">livy.server.host = 10.57.17.215</span><br><span class="line"></span><br><span class="line"># What spark master Livy sessions should use.</span><br><span class="line">livy.spark.master = spark://10.57.17.215:7077</span><br><span class="line"></span><br><span class="line"># List of local directories from where files are allowed to be added to user sessions. By</span><br><span class="line"># default it's empty, meaning users can only reference remote URIs when starting their</span><br><span class="line"># sessions.</span><br><span class="line">livy.file.local-dir-whitelist = /home/admin/apps/jars/</span><br></pre></td></tr></table></figure><p>3） 创建配置文件<code>livy-env.sh</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">cd apache-livy-0.7.0-incubating-bin/conf</span><br><span class="line">cp livy-env.sh.template livy-env.sh</span><br><span class="line">vi livy-env.sh</span><br></pre></td></tr></table></figure><p><code>livy-env.sh</code>中新增内容如下：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">export SPARK_HOME=/usr/lib/spark</span><br><span class="line">export HADOOP_CONF_DIR=/etc/hadoop/conf</span><br></pre></td></tr></table></figure><p>4） 创建配置文件<code>spark-blacklist.conf</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">cd apache-livy-0.7.0-incubating-bin/conf</span><br><span class="line">cp spark-blacklist.conf.template spark-blacklist.conf</span><br><span class="line">vi spark-blacklist.conf</span><br></pre></td></tr></table></figure><p><code>spark-blacklist.conf</code>中修改内容如下（注释掉下面2行内容），让客户端可以配置这2个参数：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"># Disallow overriding the master and the deploy mode.</span><br><span class="line">#spark.master</span><br><span class="line">#spark.submit.deployMode</span><br></pre></td></tr></table></figure><p>5）启动程序</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">cd apache-livy-0.7.0-incubating-bin</span><br><span class="line"></span><br><span class="line">./bin/livy-server start</span><br></pre></td></tr></table></figure><p>6）验证测试</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">http://10.57.17.215:8998/ui</span><br></pre></td></tr></table></figure><h4 id="三、REST调用"><a href="#三、REST调用" class="headerlink" title="三、REST调用"></a>三、REST调用</h4><p>API文档地址：<a href="http://livy.incubator.apache.org/docs/latest/rest-api.html" target="_blank" rel="noopener">http://livy.incubator.apache.org/docs/latest/rest-api.html</a></p><p>1）调用测试类</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br></pre></td><td class="code"><pre><span class="line">import com.alibaba.fastjson.JSON;</span><br><span class="line">import com.alibaba.fastjson.JSONArray;</span><br><span class="line">import com.alibaba.fastjson.JSONObject;</span><br><span class="line">import com.google.common.collect.Lists;</span><br><span class="line">import org.apache.commons.lang3.StringUtils;</span><br><span class="line"></span><br><span class="line">import java.io.IOException;</span><br><span class="line">import java.util.ArrayList;</span><br><span class="line">import java.util.HashMap;</span><br><span class="line">import java.util.List;</span><br><span class="line">import java.util.Map;</span><br><span class="line"></span><br><span class="line">/**</span><br><span class="line"> * https://www.jianshu.com/p/65d3150e8999</span><br><span class="line"> *</span><br><span class="line"> * https://blog.csdn.net/qq_36330643/article/details/76686225?locationNum=1&fps=1</span><br><span class="line"> *</span><br><span class="line"> * Q: requirement failed: Local path /data01/admin/apps/zy/transform/lib/etl.jar cannot be added to user sessions.</span><br><span class="line"> *</span><br><span class="line"> * A: 修改livy.conf： livy.file.local-dir-whitelist = /data01/admin/jars/</span><br><span class="line"> *</span><br><span class="line"> * # List of local directories from where files are allowed to be added to user sessions. By</span><br><span class="line"> # default it's empty, meaning users can only reference remote URIs when starting their</span><br><span class="line"> # sessions.</span><br><span class="line"> *</span><br><span class="line"> *</span><br><span class="line"> * @author: zhengyong Date: 2020/4/17 Time: 9:48 AM</span><br><span class="line"> */</span><br><span class="line">public class LivyTest {</span><br><span class="line"></span><br><span class="line"> public static void main(String[] args) throws IOException {</span><br><span class="line"> String url = "http://10.57.17.215:8998/batches";</span><br><span class="line"> List<Integer> sessionIds = getBatches(url);</span><br><span class="line"> deleteBatches(url, sessionIds);</span><br><span class="line"> String batches = postBatches(url);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> public static void deleteBatches(String url, List<Integer> sessionIds) throws IOException {</span><br><span class="line"> for (Integer id : sessionIds) {</span><br><span class="line"> String deleteResult = HttpConfigClient.deleteLivy(String.format("%s/%s", url, id));</span><br><span class="line"> System.out.println(String.format("delete batches id=%s, result: %s", id, deleteResult));</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> public static List<Integer> getBatches(String url) throws IOException {</span><br><span class="line"> String sessions = HttpConfigClient.getLivy(url, null);</span><br><span class="line"> JSONObject object = JSON.parseObject(sessions);</span><br><span class="line"> JSONArray array = object.getJSONArray("sessions");</span><br><span class="line"> if (array == null) {</span><br><span class="line"> return null;</span><br><span class="line"> }</span><br><span class="line"> List<Integer> sessionIds = new ArrayList<>(array.size());</span><br><span class="line"> for (Object o : array) {</span><br><span class="line"> JSONObject obj = (JSONObject) o;</span><br><span class="line"> int id = obj.getIntValue("id");</span><br><span class="line"> String name = obj.getString("name");</span><br><span class="line"> String state = obj.getString("state");</span><br><span class="line"> sessionIds.add(id);</span><br><span class="line"> System.out.println(String.format("query session id=%s, name=%s(%s)", id, name, state));</span><br><span class="line"> }</span><br><span class="line"> return sessionIds;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public static String postBatches(String url) throws IOException {</span><br><span class="line"> LivyEntity livyEntity = new LivyEntity();</span><br><span class="line"> livyEntity.setFile("/home/admin/apps/jars/etl.jar");</span><br><span class="line"> livyEntity.setClassName("cn.etl.DataEtlMain");</span><br><span class="line"> livyEntity.setNumExecutors(3);</span><br><span class="line"> livyEntity.setName("ETL");</span><br><span class="line"> livyEntity.setExecutorMemory("4g");</span><br><span class="line"> livyEntity.setExecutorCores(2);</span><br><span class="line"> livyEntity.setArgs(Lists.newArrayList("params1"));</span><br><span class="line"></span><br><span class="line"> //spark参数设置： http://spark.apache.org/docs/latest/configuration.html</span><br><span class="line"> Map<String, Object> conf = new HashMap<>();</span><br><span class="line"> conf.put("spark.master","spark://10.57.17.215:7077");</span><br><span class="line"> conf.put("spark.submit.deployMode", "cluster"); // 这一行可以把任务在spark UI上面显示；但是jar包log不能显示</span><br><span class="line"> conf.put("spark.driver.cores", "2");</span><br><span class="line"> conf.put("spark.driver.memory", "4g");</span><br><span class="line"> livyEntity.setConf(conf);</span><br><span class="line"></span><br><span class="line"> String result = HttpConfigClient.postLivy(url, JSON.toJSONString(livyEntity));</span><br><span class="line"> System.out.println("post batches result: " + result);</span><br><span class="line"> return result;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>2）httpclient工具类</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br></pre></td><td class="code"><pre><span class="line">import org.apache.commons.collections.MapUtils;</span><br><span class="line">import org.apache.commons.lang.StringUtils;</span><br><span class="line">import org.apache.http.HttpEntity;</span><br><span class="line">import org.apache.http.client.methods.CloseableHttpResponse;</span><br><span class="line">import org.apache.http.client.methods.HttpDelete;</span><br><span class="line">import org.apache.http.client.methods.HttpGet;</span><br><span class="line">import org.apache.http.client.methods.HttpPost;</span><br><span class="line">import org.apache.http.entity.ContentType;</span><br><span class="line">import org.apache.http.entity.StringEntity;</span><br><span class="line">import org.apache.http.impl.client.CloseableHttpClient;</span><br><span class="line">import org.apache.http.impl.client.HttpClients;</span><br><span class="line">import org.apache.http.util.EntityUtils;</span><br><span class="line"></span><br><span class="line">import java.io.IOException;</span><br><span class="line">import java.net.URI;</span><br><span class="line">import java.util.ArrayList;</span><br><span class="line">import java.util.Iterator;</span><br><span class="line">import java.util.Map;</span><br><span class="line"></span><br><span class="line">/**</span><br><span class="line"> *</span><br><span class="line"> */</span><br><span class="line">public class HttpConfigClient {</span><br><span class="line"> </span><br><span class="line"> public static String postLivy(String postUrl, String json) throws IOException {</span><br><span class="line"> CloseableHttpClient livyClient = HttpClients.createDefault();</span><br><span class="line"> HttpPost httpPost = new HttpPost(postUrl);</span><br><span class="line"> StringEntity entity = new StringEntity(json, ContentType.APPLICATION_JSON);</span><br><span class="line"> httpPost.setEntity(entity);</span><br><span class="line"> CloseableHttpResponse response = livyClient.execute(httpPost);</span><br><span class="line"> return handleResponse(response);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public static String getLivy(String getUrl, Map<String, Object> parametersMap) throws IOException {</span><br><span class="line"> CloseableHttpClient livyClient = HttpClients.createDefault();</span><br><span class="line"> String url = buildUri(getUrl, parametersMap).toString();</span><br><span class="line"> HttpGet httpGet = new HttpGet(url);</span><br><span class="line"> CloseableHttpResponse response = livyClient.execute(httpGet);</span><br><span class="line"> return handleResponse(response);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public static String deleteLivy(String deleteUrl) throws IOException {</span><br><span class="line"> CloseableHttpClient livyClient = HttpClients.createDefault();</span><br><span class="line"> HttpDelete httpDelete = new HttpDelete(deleteUrl);</span><br><span class="line"> CloseableHttpResponse response = livyClient.execute(httpDelete);</span><br><span class="line"> return handleResponse(response);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> /***</span><br><span class="line"> * 拼接get请求调用的URL</span><br><span class="line"> *</span><br><span class="line"> * @param url 调用请求</span><br><span class="line"> * @param parametersMap 业务数据，作为参数</span><br><span class="line"> * @return 完成url</span><br><span class="line"> */</span><br><span class="line"> private static URI buildUri(String url, Map<String, Object> parametersMap) {</span><br><span class="line"> if (MapUtils.isEmpty(parametersMap)) {</span><br><span class="line"> return URI.create(url);</span><br><span class="line"> }</span><br><span class="line"> ArrayList list = new ArrayList(parametersMap.size());</span><br><span class="line"> Iterator iterator = parametersMap.entrySet().iterator();</span><br><span class="line"></span><br><span class="line"> while (iterator.hasNext()) {</span><br><span class="line"> Map.Entry entry = (Map.Entry) iterator.next();</span><br><span class="line"> list.add(entry.getKey().toString().trim() + "=" + entry.getValue().toString().trim());</span><br><span class="line"> }</span><br><span class="line"> return list.isEmpty() ? URI.create(url) : URI.create(url + "?" + StringUtils.join(list, "&"));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * 获取结果</span><br><span class="line"> *</span><br><span class="line"> * @param response 响应</span><br><span class="line"> * @return 返回结果</span><br><span class="line"> * @throws IOException</span><br><span class="line"> */</span><br><span class="line"> private static String handleResponse(CloseableHttpResponse response) throws IOException {</span><br><span class="line"> HttpEntity resultEntity = response.getEntity();</span><br><span class="line"> return EntityUtils.toString(resultEntity);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>3）实体对象</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br><span class="line">164</span><br><span class="line">165</span><br><span class="line">166</span><br><span class="line">167</span><br><span class="line">168</span><br><span class="line">169</span><br><span class="line">170</span><br><span class="line">171</span><br><span class="line">172</span><br><span class="line">173</span><br><span class="line">174</span><br><span class="line">175</span><br><span class="line">176</span><br><span class="line">177</span><br><span class="line">178</span><br><span class="line">179</span><br><span class="line">180</span><br><span class="line">181</span><br><span class="line">182</span><br><span class="line">183</span><br><span class="line">184</span><br><span class="line">185</span><br><span class="line">186</span><br><span class="line">187</span><br><span class="line">188</span><br><span class="line">189</span><br><span class="line">190</span><br><span class="line">191</span><br><span class="line">192</span><br><span class="line">193</span><br><span class="line">194</span><br><span class="line">195</span><br><span class="line">196</span><br><span class="line">197</span><br><span class="line">198</span><br><span class="line">199</span><br><span class="line">200</span><br><span class="line">201</span><br><span class="line">202</span><br><span class="line">203</span><br><span class="line">204</span><br><span class="line">205</span><br><span class="line">206</span><br><span class="line">207</span><br><span class="line">208</span><br><span class="line">209</span><br><span class="line">210</span><br><span class="line">211</span><br><span class="line">212</span><br><span class="line">213</span><br><span class="line">214</span><br><span class="line">215</span><br><span class="line">216</span><br><span class="line">217</span><br></pre></td><td class="code"><pre><span class="line">import java.util.List;</span><br><span class="line">import java.util.Map;</span><br><span class="line"></span><br><span class="line">/**</span><br><span class="line"> * Livy REST API 封装</span><br><span class="line"> * 批处理任务 属性封装</span><br><span class="line"> */</span><br><span class="line">public class LivyEntity {</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"></span><br><span class="line"> * 必须有</span><br><span class="line"> * 包含需要执行应用的文件，主要是jar包</span><br><span class="line"> */</span><br><span class="line"> private String file;</span><br><span class="line"> /**</span><br><span class="line"> * User to impersonate when running the job</span><br><span class="line"> */</span><br><span class="line"> private String proxyUser;</span><br><span class="line"> /**</span><br><span class="line"> * Application Java/Spark main class</span><br><span class="line"> * 主类</span><br><span class="line"> */</span><br><span class="line"> private String className;</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * Command line arguments for the application</span><br><span class="line"> * 参数</span><br><span class="line"> */</span><br><span class="line"> private List<String> args;</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * jars to be used in this session</span><br><span class="line"> * 这个任务里面用到的其他 jar 包</span><br><span class="line"> */</span><br><span class="line"> private List<String> jars;</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * Python files to be used in this session</span><br><span class="line"> */</span><br><span class="line"> private List<String> pyFiles;</span><br><span class="line"> /**</span><br><span class="line"> * files to be used in this session</span><br><span class="line"> */</span><br><span class="line"> private List<String> files;</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * Amount of memory to use for the driver process</span><br><span class="line"> */</span><br><span class="line"> private String driverMemory;</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * Number of cores to use for the driver process</span><br><span class="line"> */</span><br><span class="line"> private Integer driverCores;</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * Amount of memory to use per executor process</span><br><span class="line"> */</span><br><span class="line"> private String executorMemory;</span><br><span class="line"> /**</span><br><span class="line"> * Number of cores to use for each executor</span><br><span class="line"> */</span><br><span class="line"> private Integer executorCores;</span><br><span class="line"> /**</span><br><span class="line"> * Number of executors to launch for this session</span><br><span class="line"> */</span><br><span class="line"> private Integer numExecutors;</span><br><span class="line"> /**</span><br><span class="line"> * Archives to be used in this session</span><br><span class="line"> */</span><br><span class="line"> private List<String> archives;</span><br><span class="line"> /**</span><br><span class="line"> * The name of the YARN queue to which submitted</span><br><span class="line"> */</span><br><span class="line"> private String queue;</span><br><span class="line"> /**</span><br><span class="line"> * The name of this session</span><br><span class="line"> * 任务名称</span><br><span class="line"> */</span><br><span class="line"> private String name;</span><br><span class="line"> /**</span><br><span class="line"> * Spark configuration properties</span><br><span class="line"> * spark 配置文件</span><br><span class="line"> */</span><br><span class="line"> private Map<String, Object> conf;</span><br><span class="line"></span><br><span class="line"> public String getFile() {</span><br><span class="line"> return file;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public void setFile(String file) {</span><br><span class="line"> this.file = file;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public String getProxyUser() {</span><br><span class="line"> return proxyUser;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public void setProxyUser(String proxyUser) {</span><br><span class="line"> this.proxyUser = proxyUser;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public String getClassName() {</span><br><span class="line"> return className;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public void setClassName(String className) {</span><br><span class="line"> this.className = className;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public List<String> getArgs() {</span><br><span class="line"> return args;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public void setArgs(List<String> args) {</span><br><span class="line"> this.args = args;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public List<String> getJars() {</span><br><span class="line"> return jars;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public void setJars(List<String> jars) {</span><br><span class="line"> this.jars = jars;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public List<String> getPyFiles() {</span><br><span class="line"> return pyFiles;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public void setPyFiles(List<String> pyFiles) {</span><br><span class="line"> this.pyFiles = pyFiles;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public List<String> getFiles() {</span><br><span class="line"> return files;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public void setFiles(List<String> files) {</span><br><span class="line"> this.files = files;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public String getDriverMemory() {</span><br><span class="line"> return driverMemory;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public void setDriverMemory(String driverMemory) {</span><br><span class="line"> this.driverMemory = driverMemory;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public Integer getDriverCores() {</span><br><span class="line"> return driverCores;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public void setDriverCores(Integer driverCores) {</span><br><span class="line"> this.driverCores = driverCores;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public String getExecutorMemory() {</span><br><span class="line"> return executorMemory;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public void setExecutorMemory(String executorMemory) {</span><br><span class="line"> this.executorMemory = executorMemory;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public Integer getExecutorCores() {</span><br><span class="line"> return executorCores;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public void setExecutorCores(Integer executorCores) {</span><br><span class="line"> this.executorCores = executorCores;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public Integer getNumExecutors() {</span><br><span class="line"> return numExecutors;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public void setNumExecutors(Integer numExecutors) {</span><br><span class="line"> this.numExecutors = numExecutors;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public List<String> getArchives() {</span><br><span class="line"> return archives;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public void setArchives(List<String> archives) {</span><br><span class="line"> this.archives = archives;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public String getQueue() {</span><br><span class="line"> return queue;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public void setQueue(String queue) {</span><br><span class="line"> this.queue = queue;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public String getName() {</span><br><span class="line"> return name;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public void setName(String name) {</span><br><span class="line"> this.name = name;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public Map<String, Object> getConf() {</span><br><span class="line"> return conf;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public void setConf(Map<String, Object> conf) {</span><br><span class="line"> this.conf = conf;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure><h4 id="四、异常问题"><a href="#四、异常问题" class="headerlink" title="四、异常问题"></a>四、异常问题</h4><p>问题：requirement failed: Local path /home/admin/apps/jars/etl.jar cannot be added to user sessions.</p><p>解决办法：修改<code>livy.conf</code>配置</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">livy.file.local-dir-whitelist = /home/admin/apps/jars/</span><br></pre></td></tr></table></figure><h4 id="五、代码地址"><a href="#五、代码地址" class="headerlink" title="五、代码地址"></a>五、代码地址</h4><pre><code>https://github.com/zyongjava/pomelo/tree/master/src/main/java/livy</code></pre>]]></content>
<summary type="html">
<h3 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h3><pre><code>本次部署是在单台机器10.57.17.215上部署，只部署了spark和livy；通过配置livy加载本地文件jar；然后通
</summary>
</entry>
<entry>
<title>HIVE部署</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2019/12/24/HIVE%E9%83%A8%E7%BD%B2/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2019/12/24/HIVE部署/</id>
<published>2019-12-24T03:08:13.000Z</published>
<updated>2019-12-24T06:26:08.966Z</updated>
<content type="html"><![CDATA[<h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><p>部署hive需要提前部署好以下环境：</p><ol><li>mysql （安装在10.57.22.113）</li><li>HDFS (hadoop-2.7.7)</li></ol><h4 id="一、搭建模式"><a href="#一、搭建模式" class="headerlink" title="一、搭建模式"></a>一、搭建模式</h4><p>Hive中搭建分如下3中方式，三种方式归根到底就是元数据的存储位置不一样</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">a)内嵌Derby方式 ：运行hive会在当前目录生成一个derby文件和一个metastore_db目录。这种存储方式的弊端是在同一个目录下同时只能有一个hive客户端能使用数据库。</span><br><span class="line">b)mysql Local方式 (hive.metastore.uris为空表示嵌入模式或本地模式)</span><br><span class="line">c)多用户模式 (mysql Remote方式; hive.metastore.uris不为空为远程模式)</span><br></pre></td></tr></table></figure><h4 id="二、部署Hive"><a href="#二、部署Hive" class="headerlink" title="二、部署Hive"></a>二、部署Hive</h4><p>A）下载地址：<code>https://archive.apache.org/dist/hive/hive-2.3.4/apache-hive-2.3.4-bin.tar.gz</code></p><p>B）解压包：<code>tar -xvf apache-hive-2.3.4-bin.tar.gz</code></p><p>C）修改文件夹名称：<code>mv apache-hive-2.3.4-bin hive-2.3.4</code></p><p>D）配置环境变量, 修改<code>.bashrc</code>文件</p><p>vi ~/.bashrc</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">export HIVE_HOME=/data01/bigdata/hive-2.3.4</span><br><span class="line">export PATH=$PATH:$HIVE_HOME/bin</span><br></pre></td></tr></table></figure><p>source ~/.bashrc</p><h4 id="三、配置修改"><a href="#三、配置修改" class="headerlink" title="三、配置修改"></a>三、配置修改</h4><p>A）修改hive-site.xml配置</p><p>首先复制一份配置文件<code>cp hive-default.xml.template hive-site.xml</code>，修改如下配置：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br></pre></td><td class="code"><pre><span class="line"><!-- 指定HDFS中的hive仓库地址 --> </span><br><span class="line"> <property></span><br><span class="line"> <name>hive.metastore.warehouse.dir</name></span><br><span class="line"> <value>/data01/bigdata/hive-2.3.4/warehouse</value></span><br><span class="line"> <description>location of default database for the warehouse</description></span><br><span class="line"> </property></span><br><span class="line"> </span><br><span class="line"> <property></span><br><span class="line"> <name>hive.exec.scratchdir</name></span><br><span class="line"> <value>/data01/bigdata/hive-2.3.4</value></span><br><span class="line"> <description>HDFS root scratch dir for Hive jobs which gets created with write all (733) permission. For each connecting user, an HDFS scratch dir: ${hive.exec.scratchdir}/&lt;username&gt; is created, with ${hive.scratch.dir.permission}.</description></span><br><span class="line"> </property></span><br><span class="line"></span><br><span class="line"><!-- 指定驱动类 --></span><br><span class="line"> <property></span><br><span class="line"> <name>javax.jdo.option.ConnectionDriverName</name></span><br><span class="line"> <value>com.mysql.jdbc.Driver</value></span><br><span class="line"> <description>Driver class name for a JDBC metastore</description></span><br><span class="line"> </property></span><br><span class="line"></span><br><span class="line"> <property></span><br><span class="line"> <name>javax.jdo.option.ConnectionURL</name></span><br><span class="line"> <value>jdbc:mysql://10.57.22.113/hive_remote?createDatabaseIfNotExist=true</value></span><br><span class="line"> <description></span><br><span class="line"> JDBC connect string for a JDBC metastore.</span><br><span class="line"> To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.</span><br><span class="line"> For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.</span><br><span class="line"> </description></span><br><span class="line"></span><br><span class="line"> <property></span><br><span class="line"> <name>javax.jdo.option.ConnectionUserName</name></span><br><span class="line"> <value>root</value></span><br><span class="line"> <description>Username to use against metastore database</description></span><br><span class="line"> </property></span><br><span class="line"></span><br><span class="line"> <property></span><br><span class="line"> <name>javax.jdo.option.ConnectionPassword</name></span><br><span class="line"> <value>root</value></span><br><span class="line"> <description>password to use against metastore database</description></span><br><span class="line"> </property></span><br><span class="line"></span><br><span class="line"><!-- 该属性为空表示嵌入模式或本地模式，否则为远程模式 --> </span><br><span class="line"> <property></span><br><span class="line"> <name>hive.metastore.uris</name></span><br><span class="line"> <value>thrift://10.57.22.119:9083</value></span><br><span class="line"> <description>Thrift URI for the remote metastore. Used by metastore client to connect to remote metastore.</description></span><br><span class="line"> </property></span><br><span class="line"> </span><br><span class="line"> <property></span><br><span class="line"> <name>hive.exec.local.scratchdir</name></span><br><span class="line"> <value>/tmp/hive/local</value></span><br><span class="line"> <description>Local scratch space for Hive jobs</description></span><br><span class="line"> </property></span><br><span class="line"> </span><br><span class="line"> <property></span><br><span class="line"> <name>hive.downloaded.resources.dir</name></span><br><span class="line"> <value>/tmp/hive/resources</value></span><br><span class="line"> <description>Temporary local directory for added resources in the remote file system.</description></span><br><span class="line"> </property></span><br><span class="line"> </span><br><span class="line"><property></span><br><span class="line"> <name>hive.server2.webui.host</name></span><br><span class="line"> <value>tdcdh119</value></span><br><span class="line"> <description>The host address the HiveServer2 WebUI will listen on</description></span><br><span class="line"> </property></span><br><span class="line"> <property></span><br><span class="line"> <name>hive.server2.webui.port</name></span><br><span class="line"> <value>10002</value></span><br><span class="line"> <description>The port the HiveServer2 WebUI will listen on. This can beset to 0 or a negative integer to disable the web UI</description></span><br><span class="line"> </property></span><br></pre></td></tr></table></figure><p>B) 修改hive-env.sh变量</p><p>首先复制一份配置文件<code>cp hive-env.sh.template hive-env.sh</code>, 新增如下配置：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">export HADOOP_HOME=/data01/bigdata/hive-2.3.4</span><br><span class="line">export HIVE_CONF_DIR=/data01/bigdata/hive-2.3.4/conf</span><br><span class="line">export HIVE_AUX_JARS_PATH=/data01/bigdata/hive-2.3.4lib</span><br></pre></td></tr></table></figure><h4 id="四、其他配置"><a href="#四、其他配置" class="headerlink" title="四、其他配置"></a>四、其他配置</h4><p>A） 新建warehouse目录</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">mkdir /data01/bigdata/hive-2.3.4/warehouse</span><br><span class="line"></span><br><span class="line">将warehouse变为hadoop仓库</span><br><span class="line">hadoop fs -mkdir -p /data01/bigdata/hive-2.3.4/warehouse</span><br><span class="line">hadoop fs -chmod 777 /data01/bigdata/hive-2.3.4/warehouse</span><br></pre></td></tr></table></figure><p>B）上传<code>mysql-connector-java-5.1.40.jar</code>到指定目录<code>$HADOOP_HOME/lib</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">scp ~/Downloads/mysql-connector-java-5.1.40.jar admin@10.57.22.119:/data01/bigdata/hive-2.3.4/lib</span><br></pre></td></tr></table></figure><p>C) 创建存储元数据的mysql库</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">mysql -h 10.57.22.113 -u root -p</span><br><span class="line">create database hive_remote;</span><br></pre></td></tr></table></figure><p>D) 初始化schema（<code>/data01/bigdata/hive-2.3.4/bin/schematool</code>）</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">schematool -dbType mysql -initSchema</span><br></pre></td></tr></table></figure><p>进入mysql查看初始化schema后结果，如下：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br></pre></td><td class="code"><pre><span class="line">mysql> use hive_remote;</span><br><span class="line">Reading table information for completion of table and column names</span><br><span class="line">You can turn off this feature to get a quicker startup with -A</span><br><span class="line"></span><br><span class="line">Database changed</span><br><span class="line">mysql> show tables;</span><br><span class="line">+---------------------------+</span><br><span class="line">| Tables_in_hive_remote |</span><br><span class="line">+---------------------------+</span><br><span class="line">| aux_table |</span><br><span class="line">| bucketing_cols |</span><br><span class="line">| cds |</span><br><span class="line">| columns_v2 |</span><br><span class="line">| compaction_queue |</span><br><span class="line">| completed_compactions |</span><br><span class="line">| completed_txn_components |</span><br><span class="line">| database_params |</span><br><span class="line">| db_privs |</span><br><span class="line">| dbs |</span><br><span class="line">| delegation_tokens |</span><br><span class="line">| func_ru |</span><br><span class="line">| funcs |</span><br><span class="line">| global_privs |</span><br><span class="line">| hive_locks |</span><br><span class="line">| idxs |</span><br><span class="line">| index_params |</span><br><span class="line">| key_constraints |</span><br><span class="line">| master_keys |</span><br><span class="line">| next_compaction_queue_id |</span><br><span class="line">| next_lock_id |</span><br><span class="line">| next_txn_id |</span><br><span class="line">| notification_log |</span><br><span class="line">| notification_sequence |</span><br><span class="line">| nucleus_tables |</span><br><span class="line">| part_col_privs |</span><br><span class="line">| part_col_stats |</span><br><span class="line">| part_privs |</span><br><span class="line">| partition_events |</span><br><span class="line">| partition_key_vals |</span><br><span class="line">| partition_keys |</span><br><span class="line">| partition_params |</span><br><span class="line">| partitions |</span><br><span class="line">| role_map |</span><br><span class="line">| roles |</span><br><span class="line">| sd_params |</span><br><span class="line">| sds |</span><br><span class="line">| sequence_table |</span><br><span class="line">| serde_params |</span><br><span class="line">| serdes |</span><br><span class="line">| skewed_col_names |</span><br><span class="line">| skewed_col_value_loc_map |</span><br><span class="line">| skewed_string_list |</span><br><span class="line">| skewed_string_list_values |</span><br><span class="line">| skewed_values |</span><br><span class="line">| sort_cols |</span><br><span class="line">| tab_col_stats |</span><br><span class="line">| table_params |</span><br><span class="line">| tbl_col_privs |</span><br><span class="line">| tbl_privs |</span><br><span class="line">| tbls |</span><br><span class="line">| txn_components |</span><br><span class="line">| txns |</span><br><span class="line">| type_fields |</span><br><span class="line">| types |</span><br><span class="line">| version |</span><br><span class="line">| write_set |</span><br><span class="line">+---------------------------+</span><br><span class="line">57 rows in set (0.06 sec)</span><br></pre></td></tr></table></figure><h4 id="五、启动hive-server"><a href="#五、启动hive-server" class="headerlink" title="五、启动hive server"></a>五、启动hive server</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">启动 hive server</span><br><span class="line">nohup hive --service metastore &</span><br><span class="line"></span><br><span class="line">启动hive web UI</span><br><span class="line">nohup hive --service hiveserver2 &</span><br></pre></td></tr></table></figure><p>进程查看,是否有<code>RunJar</code>进程（有即服务已启动）：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">[admin@tdcdh119 warehouse]$ jps</span><br><span class="line">17627 RunJar</span><br><span class="line">19643 Jps</span><br></pre></td></tr></table></figure><h4 id="六、启动hive-client"><a href="#六、启动hive-client" class="headerlink" title="六、启动hive client"></a>六、启动hive client</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">hive</span><br></pre></td></tr></table></figure><p>验证hive</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">hive> create databases hive_test;</span><br><span class="line">OK</span><br><span class="line">Time taken: 1.184 seconds</span><br><span class="line">hive> show databases;</span><br><span class="line">OK</span><br><span class="line">default</span><br><span class="line">hive_test</span><br><span class="line">Time taken: 1.131 seconds, Fetched: 2 row(s)</span><br></pre></td></tr></table></figure><h4 id="七、参考文档"><a href="#七、参考文档" class="headerlink" title="七、参考文档"></a>七、参考文档</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-InstallingHivefromaStableRelease</span><br><span class="line"></span><br><span class="line">https://www.cnblogs.com/LHWorldBlog/p/8289702.html</span><br><span class="line"></span><br><span class="line">https://www.cnblogs.com/dxxblog/p/8193967.html</span><br><span class="line"></span><br><span class="line">https://www.cnblogs.com/xuwujing/p/8045821.html</span><br><span class="line"></span><br><span class="line">https://blog.csdn.net/zhangxiaohui4445/article/details/90675445</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<h4 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h4><p>部署hive需要提前部署好以下环境：</p>
<ol>
<li>mysql （安装在10.57.22.113）</li>
<li>HDFS
</summary>
<category term="hdfs" scheme="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/tags/hdfs/"/>
</entry>
<entry>
<title>Spark集群部署</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2019/12/24/Spark%E9%9B%86%E7%BE%A4%E9%83%A8%E7%BD%B2/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2019/12/24/Spark集群部署/</id>
<published>2019-12-24T03:05:54.000Z</published>
<updated>2019-12-24T03:31:54.591Z</updated>
<content type="html"><![CDATA[<h3 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h3><p>本次基于Standalone的Spark集群部署使用3台服务器，分布如下：</p><table><thead><tr><th>服务器地址</th><th>hostname</th><th>Master</th><th>Worker</th></tr></thead><tbody><tr><td>10.57.22.116</td><td>tdcdh116</td><td>no</td><td>yes</td></tr><tr><td>10.57.22.117</td><td>tdcdh117</td><td>no</td><td>yes</td></tr><tr><td>10.57.22.119</td><td>tdcdh119</td><td>yes</td><td>yes</td></tr></tbody></table><h4 id="一、部署"><a href="#一、部署" class="headerlink" title="一、部署"></a>一、部署</h4><p>下载地址：<code>https://archive.apache.org/dist/spark/spark-2.3.4/</code></p><p>A. 解压安装包</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">tar -xvf spark-2.3.4-bin-hadoop2.7.tgz</span><br></pre></td></tr></table></figure><p>B. 更改包名称为<code>spark-2.3.4</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">mv spark-2.3.4-bin-hadoop2.7/spark-2.3.4</span><br></pre></td></tr></table></figure><p>C. 配置slaves（即Worker主机）</p><p>1）进入包目录：<code>/data01/bigdata/spark-2.3.4/conf</code></p><p>2) 复制文件：<code>cp slaves.template slaves</code></p><p>3) 修改文件：<code>vi slaves</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">tdcdh116</span><br><span class="line">tdcdh117</span><br><span class="line">tdcdh119</span><br></pre></td></tr></table></figure><p>D. 配置spark-env.sh</p><p>1）新建work目录：<code>mkdir /data01/bigdata/spark-2.3.4/work</code></p><p>2) 复制文件：<code>cp spark-env.template spark-env.sh</code></p><p>3) 修改文件：<code>vi spark-env.sh</code></p><p>新增如下配置：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">#主节点ip或hostname</span><br><span class="line">export SPARK_MASTER_HOST=tdcdh119</span><br><span class="line">#主节点的端口</span><br><span class="line">export SPARK_MASTER_PORT=7077</span><br><span class="line">#worker的工作目录区</span><br><span class="line">export SPARK_WORKER_DIR=/data01/bigdata/spark-2.3.4/work</span><br><span class="line">export SPARK_WORKER_MEMORY=1g</span><br><span class="line">export SPARK_WORKER_CORES=2</span><br><span class="line">export JAVA_HOME=/data01/admin/local/jdk8u191</span><br><span class="line">export HADOOP_HOME=/data01/bigdata/hadoop-2.7.7</span><br><span class="line">export HADOOP_CONF_DIR=/data01/bigdata/hadoop-2.7.7/etc/hadoop</span><br></pre></td></tr></table></figure><p>E. 将包拷贝到所有机器</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">scp -r spark-2.3.4 admin@10.57.22.116:/data01/bigdata/</span><br><span class="line">scp -r spark-2.3.4 admin@10.57.22.117:/data01/bigdata/</span><br></pre></td></tr></table></figure><p>F. 环境变量配置</p><p>1） 修改sudo vi ~/.bashrc</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">export SPARK_HOME=/data01/bigdata/spark-2.3.4</span><br><span class="line">export PATH=$SPARK_HOME/bin:$SPARK_HOME/sbin:$PATH</span><br></pre></td></tr></table></figure><p>2）生效：source ~/.bashrc</p><p>G. 启动项目</p><p>只需要在Master(10.57.22.119)节点启动即可</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">cd /data01/bigdata/spark-2.3.4</span><br><span class="line">sbin/start-all.sh</span><br></pre></td></tr></table></figure><h4 id="二、项目验证"><a href="#二、项目验证" class="headerlink" title="二、项目验证"></a>二、项目验证</h4><p><a href="http://10.57.22.119:8080/" target="_blank" rel="noopener">http://10.57.22.119:8080/</a></p><h4 id="三、参考文档"><a href="#三、参考文档" class="headerlink" title="三、参考文档"></a>三、参考文档</h4><p><a href="https://blog.csdn.net/qq_42825815/article/details/84071702">https://blog.csdn.net/qq_42825815/article/details/84071702</a><br><a href="https://www.cnblogs.com/lyy-blog/p/9636070.html" target="_blank" rel="noopener">https://www.cnblogs.com/lyy-blog/p/9636070.html</a></p><h4 id="四、备注"><a href="#四、备注" class="headerlink" title="四、备注"></a>四、备注</h4><p>需要免密登录, 参考《HDFS集群部署》</p>]]></content>
<summary type="html">
<h3 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h3><p>本次基于Standalone的Spark集群部署使用3台服务器，分布如下：</p>
<table>
<thead>
<tr>
<th>服务器
</summary>
</entry>
<entry>
<title>HDFS集群部署</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2019/12/24/HDFS%E9%9B%86%E7%BE%A4%E9%83%A8%E7%BD%B2/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2019/12/24/HDFS集群部署/</id>
<published>2019-12-24T03:04:37.000Z</published>
<updated>2019-12-24T03:31:48.732Z</updated>
<content type="html"><![CDATA[<h3 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h3><p>本次部署使用3台服务器，分布如下：</p><table><thead><tr><th>服务器地址</th><th>hostname</th><th>NameNode</th><th>DataNode</th></tr></thead><tbody><tr><td>10.57.22.116</td><td>tdcdh116</td><td>no</td><td>yes</td></tr><tr><td>10.57.22.117</td><td>tdcdh117</td><td>no</td><td>yes</td></tr><tr><td>10.57.22.119</td><td>tdcdh119</td><td>yes</td><td>yes</td></tr></tbody></table><p>下载地址： <a href="https://archive.apache.org/dist/hadoop/common/hadoop-2.7.7/" target="_blank" rel="noopener">https://archive.apache.org/dist/hadoop/common/hadoop-2.7.7/</a></p><h3 id="一、同步服务器时间"><a href="#一、同步服务器时间" class="headerlink" title="一、同步服务器时间"></a>一、同步服务器时间</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">因为安装centos镜像时的选项不同，可能会导致我们各个节点的系统时间不一致，这会导致接收到的信息因为时间相差过大而被认为是过时信息。为了同步各个节点之间的时间，我们借用ntp工具</span><br><span class="line"></span><br><span class="line">sudo yum install ntp</span><br><span class="line"></span><br><span class="line">然后我们可以去往上搜索那些开放的时间服务器，这里我们采用阿里云的时间服务器</span><br><span class="line"></span><br><span class="line">sudo ntpdate ntp1.aliyun.com</span><br><span class="line"></span><br><span class="line">同步完成后我们可以在终端打印时间信息</span><br><span class="line"></span><br><span class="line">date</span><br></pre></td></tr></table></figure><h3 id="二、修改hostname"><a href="#二、修改hostname" class="headerlink" title="二、修改hostname"></a>二、修改hostname</h3><p>1) 修改主机名</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">sudo hostname tdcdh119</span><br></pre></td></tr></table></figure><p>2) 修改hosts</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">sudo vi /etc/hosts</span><br><span class="line"></span><br><span class="line">10.57.22.116 tdcdh116</span><br><span class="line">10.57.22.117 tdcdh117</span><br><span class="line">10.57.22.119 tdcdh119</span><br><span class="line"></span><br><span class="line">source /etc/hosts</span><br></pre></td></tr></table></figure><p>3) 关闭防火墙</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">sudo systemctl stop firewalld</span><br></pre></td></tr></table></figure><h3 id="三、配置免密登录"><a href="#三、配置免密登录" class="headerlink" title="三、配置免密登录"></a>三、配置免密登录</h3><p>a. 查看本地是否有秘钥：执行命令<code>ls ~/.ssh</code>查看是否有文件<code>id_rsa</code>和<code>id_rsa.pub</code></p><p>b. 生成秘钥：如果没有看到<code>a</code>步骤中的两个文件则执行下面命令生成公钥和私钥<code>ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa</code></p><p>c. 免密设置：第a，b步是在所有节点执行，当前步骤只在主节点<code>(本文档为10.57.22.119节点)</code>执行如下命令：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">ssh-copy-id -i ~/.ssh/id_rsa.pub admin@tdcdh116</span><br><span class="line">ssh-copy-id -i ~/.ssh/id_rsa.pub admin@tdcdh117</span><br><span class="line">ssh-copy-id -i ~/.ssh/id_rsa.pub admin@tdcdh119</span><br></pre></td></tr></table></figure><p>d. 验证免密是否配置成功: 在10.57.22.116机器上执行命令<code>ssh admin@tdcdh119</code>, 如果不需要密码就能登录证明成功；</p><h3 id="四、安装Hadoop"><a href="#四、安装Hadoop" class="headerlink" title="四、安装Hadoop"></a>四、安装Hadoop</h3><p>A). 创建数据目录和临时目录:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">cd hadoop-2.7.7</span><br><span class="line"></span><br><span class="line">mkdir datas tmp datas/data datas/name</span><br></pre></td></tr></table></figure><p>B). 配置JAVA_HOME</p><p>vi /hadoop-2.7.7/etc/hadoop/hadoop-env.sh</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">export JAVA_HOME=/data01/admin/local/jdk8u191</span><br></pre></td></tr></table></figure><p>C). 修改配置文件</p><p>1） hdfs-site.xml </p><p>10.57.22.116配置如下：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><property></span><br><span class="line"> <name>dfs.replication</name></span><br><span class="line"> <value>2</value></span><br><span class="line"></property></span><br><span class="line"></span><br><span class="line"><property></span><br><span class="line"> <name>dfs.namenode.secondary.http-address</name></span><br><span class="line"> <value>tdcdh116:50090</value></span><br><span class="line"></property></span><br><span class="line"></span><br><span class="line"><property></span><br><span class="line"><name>dfs.datanode.data.dir</name></span><br><span class="line"><value>/data01/bigdata/hadoop-2.7.7/datas/data</value></span><br><span class="line"></property></span><br></pre></td></tr></table></figure><p>10.57.22.117配置如下：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><property></span><br><span class="line"> <name>dfs.replication</name></span><br><span class="line"> <value>2</value></span><br><span class="line"></property></span><br><span class="line"></span><br><span class="line"><property></span><br><span class="line"> <name>dfs.namenode.secondary.http-address</name></span><br><span class="line"> <value>tdcdh116:50090</value></span><br><span class="line"></property></span><br><span class="line"></span><br><span class="line"><property></span><br><span class="line"><name>dfs.datanode.data.dir</name></span><br><span class="line"><value>/data01/bigdata/hadoop-2.7.7/datas/data</value></span><br><span class="line"></property></span><br></pre></td></tr></table></figure><p>10.57.22.119配置如下：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><property></span><br><span class="line"> <name>dfs.replication</name></span><br><span class="line"> <value>2</value></span><br><span class="line"></property></span><br><span class="line"></span><br><span class="line"><!-- second namenode节点 --></span><br><span class="line"><property></span><br><span class="line"> <name>dfs.namenode.secondary.http-address</name></span><br><span class="line"> <value>tdcdh116:50090</value></span><br><span class="line"></property></span><br><span class="line"></span><br><span class="line"><!-- namenode节点 --></span><br><span class="line"><property></span><br><span class="line"><name>dfs.namenode.name.dir</name></span><br><span class="line"><value>/data01/bigdata/hadoop-2.7.7/datas/name</value></span><br><span class="line"></property></span><br><span class="line"></span><br><span class="line"><!-- datanode节点 --></span><br><span class="line"><property></span><br><span class="line"><name>dfs.datanode.data.dir</name></span><br><span class="line"><value>/data01/bigdata/hadoop-2.7.7/datas/data</value></span><br><span class="line"></property></span><br><span class="line"></span><br><span class="line"><!-- 开启webhdfs --></span><br><span class="line"><property> </span><br><span class="line"><name>dfs.webhdfs.enabled</name> </span><br><span class="line"><value>true</value> </span><br><span class="line"></property></span><br></pre></td></tr></table></figure><p>2）core-site.xml</p><p>10.57.22.116、10.57.22.117、10.57.22.119配置均如下：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><!-- 指向namenode节点 --></span><br><span class="line"><property></span><br><span class="line"> <name>fs.defaultFS</name></span><br><span class="line"> <value>hdfs://tdcdh119:9000</value></span><br><span class="line"></property></span><br><span class="line"></span><br><span class="line"><property></span><br><span class="line"> <name>hadoop.tmp.dir</name></span><br><span class="line"> <value>/data01/bigdata/hadoop-2.7.7/tmp</value></span><br><span class="line"> </property></span><br></pre></td></tr></table></figure><p>3) 修改slaves（即datanode节点）</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">tdcdh116</span><br><span class="line">tdcdh117</span><br><span class="line">tdcdh119</span><br></pre></td></tr></table></figure><p>D）第一次启动前格式化namenode</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">cd /hadoop-2.7.7</span><br><span class="line"></span><br><span class="line">bin/hdfs namenode -format</span><br></pre></td></tr></table></figure><p>E）只需要在10.57.22.119节点启动就启动了整个集群（即就把其他2台全部启动了）：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">cd /hadoop-2.7.7</span><br><span class="line"></span><br><span class="line">sbin/start-dfs.sh</span><br></pre></td></tr></table></figure><h3 id="五、环境变量配置"><a href="#五、环境变量配置" class="headerlink" title="五、环境变量配置"></a>五、环境变量配置</h3><p>1） 修改sudo vi ~/.bashrc</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">export HADOOP_HOME=/data01/bigdata/hadoop-2.7.7</span><br><span class="line">export PATH=${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin:$PATH</span><br></pre></td></tr></table></figure><p>2）生效：source ~/.bashrc</p><h3 id="六、访问地址"><a href="#六、访问地址" class="headerlink" title="六、访问地址"></a>六、访问地址</h3><p><a href="http://10.57.22.119:50070/" target="_blank" rel="noopener">http://10.57.22.119:50070/</a></p><h3 id="七、参考文档"><a href="#七、参考文档" class="headerlink" title="七、参考文档"></a>七、参考文档</h3><p><a href="https://blog.csdn.net/qq_40246546/article/details/83003389">https://blog.csdn.net/qq_40246546/article/details/83003389</a></p>]]></content>
<summary type="html">
<h3 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h3><p>本次部署使用3台服务器，分布如下：</p>
<table>
<thead>
<tr>
<th>服务器地址</th>
<th>hostname
</summary>
<category term="hadoop" scheme="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/tags/hadoop/"/>
</entry>
<entry>
<title>经典技术文章概览</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/12/28/%E7%BB%8F%E5%85%B8%E6%8A%80%E6%9C%AF%E6%96%87%E7%AB%A0%E6%A6%82%E8%A7%88/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/12/28/经典技术文章概览/</id>
<published>2018-12-28T07:34:58.000Z</published>
<updated>2020-10-12T06:41:41.214Z</updated>
<content type="html"><![CDATA[<p>以下是我平时阅读发现比较有深度的文章；做个记录</p><table><thead><tr><th>文章标题</th><th>地址</th></tr></thead><tbody><tr><td>RPC原理</td><td><a href="http://www.cnblogs.com/LBSer/p/4853234.html" target="_blank" rel="noopener">http://www.cnblogs.com/LBSer/p/4853234.html</a></td></tr><tr><td>NIO理解</td><td><a href="http://www.cnblogs.com/LBSer/p/4622749.html" target="_blank" rel="noopener">http://www.cnblogs.com/LBSer/p/4622749.html</a></td></tr><tr><td>Netty系列之Netty高性能之道</td><td><a href="http://www.infoq.com/cn/articles/netty-high-performance" target="_blank" rel="noopener">http://www.infoq.com/cn/articles/netty-high-performance</a></td></tr><tr><td>自顶向下深入分析Netty</td><td><a href="https://www.jianshu.com/p/da4398743b5a" target="_blank" rel="noopener">https://www.jianshu.com/p/da4398743b5a</a><br> <a href="https://www.jianshu.com/p/92c5092c5d3f" target="_blank" rel="noopener">https://www.jianshu.com/p/92c5092c5d3f</a></td></tr><tr><td>分布式系统数据一致性</td><td><a href="http://chuansong.me/n/286764951149" target="_blank" rel="noopener">http://chuansong.me/n/286764951149</a></td></tr><tr><td>Java 并发开发,内置锁 Synchronized</td><td><a href="http://www.kuqin.com/shuoit/20170217/353325.html?url_type=39&object_type=webpage&pos=1" target="_blank" rel="noopener">http://www.kuqin.com/shuoit/20170217/353325.html?url_type=39&object_type=webpage&pos=1</a></td></tr><tr><td>Mysql聚簇索引和非聚簇索引原理</td><td><a href="http://blog.csdn.net/lisuyibmd/article/details/53004848">http://blog.csdn.net/lisuyibmd/article/details/53004848</a></td></tr><tr><td>大顶堆排序</td><td><a href="http://m.blog.csdn.net/article/details?id=19039931" target="_blank" rel="noopener">http://m.blog.csdn.net/article/details?id=19039931</a></td></tr><tr><td>Junit mockito 解耦合测试</td><td><a href="https://mp.weixin.qq.com/s?__biz=MjM5NzMyMjAwMA==&mid=2651478581&idx=2&sn=0b40e885c2fb81be865a2f7567713a3f&chksm=bd25364a8a52bf5c6fc2f13af0429f8e07b60a47a07e95a657e1339aac229ae4ab5a3f54b059&mpshare=1&scene=1&srcid=05249JIFI9paA3zVLBVrqi9g#rd" target="_blank" rel="noopener">https://mp.weixin.qq.com/s?__biz=MjM5NzMyMjAwMA==&mid=2651478581&idx=2&sn=0b40e885c2fb81be865a2f7567713a3f&chksm=bd25364a8a52bf5c6fc2f13af0429f8e07b60a47a07e95a657e1339aac229ae4ab5a3f54b059&mpshare=1&scene=1&srcid=05249JIFI9paA3zVLBVrqi9g#rd</a></td></tr><tr><td>Dubbo中消费者初始化的过程解析</td><td><a href="http://cxis.me/2017/03/21/Dubbo%E4%B8%AD%E6%B6%88%E8%B4%B9%E8%80%85%E5%88%9D%E5%A7%8B%E5%8C%96%E7%9A%84%E8%BF%87%E7%A8%8B%E8%A7%A3%E6%9E%90/?utm_source=tuicool&utm_medium=referral" target="_blank" rel="noopener">http://cxis.me/2017/03/21/Dubbo%E4%B8%AD%E6%B6%88%E8%B4%B9%E8%80%85%E5%88%9D%E5%A7%8B%E5%8C%96%E7%9A%84%E8%BF%87%E7%A8%8B%E8%A7%A3%E6%9E%90/?utm_source=tuicool&utm_medium=referral</a></td></tr><tr><td>海量数据处理算法—Bloom Filter</td><td><a href="http://blog.csdn.net/hguisu/article/details/7866173">http://blog.csdn.net/hguisu/article/details/7866173</a></td></tr><tr><td>Kafka剖析（一）：Kafka背景及架构介绍</td><td><a href="http://www.infoq.com/cn/articles/kafka-analysis-part-1" target="_blank" rel="noopener">http://www.infoq.com/cn/articles/kafka-analysis-part-1</a></td></tr><tr><td>一个java对象到底内存有多大</td><td><a href="http://www.cnblogs.com/zhanjindong/p/3757767.html" target="_blank" rel="noopener">http://www.cnblogs.com/zhanjindong/p/3757767.html</a></td></tr><tr><td>关于分布式事务、两阶段提交协议、三阶提交协议</td><td><a href="http://blog.jobbole.com/95632" target="_blank" rel="noopener">http://blog.jobbole.com/95632</a></td></tr><tr><td>java知识分享</td><td><a href="https://github.com/crossoverJie/Java-Interview" target="_blank" rel="noopener">https://github.com/crossoverJie/Java-Interview</a></td></tr><tr><td>spring集成mybatis实现mysql读写分离</td><td><a href="https://www.cnblogs.com/lidj/p/7337535.html" target="_blank" rel="noopener">https://www.cnblogs.com/lidj/p/7337535.html</a></td></tr><tr><td>DDD-领域驱动设计在互联网业务开发中的实践</td><td><a href="https://zhuanlan.zhihu.com/p/32459776" target="_blank" rel="noopener">https://zhuanlan.zhihu.com/p/32459776</a></td></tr><tr><td>注释驱动的 Spring cache 缓存介绍</td><td><a href="https://www.ibm.com/developerworks/cn/opensource/os-cn-spring-cache/index.html" target="_blank" rel="noopener">https://www.ibm.com/developerworks/cn/opensource/os-cn-spring-cache/index.html</a></td></tr><tr><td>Tomcat系统架构</td><td><a href="https://blog.csdn.net/xlgen157387/article/details/79006434">https://blog.csdn.net/xlgen157387/article/details/79006434</a></td></tr><tr><td>Tomcat工作原理</td><td><a href="https://www.ibm.com/developerworks/cn/java/j-lo-tomcat1" target="_blank" rel="noopener">https://www.ibm.com/developerworks/cn/java/j-lo-tomcat1</a></td></tr><tr><td>图解Tomcat类加载机制</td><td><a href="https://www.aliyun.com/jiaocheng/290132.html" target="_blank" rel="noopener">https://www.aliyun.com/jiaocheng/290132.html</a></td></tr><tr><td>深入理解Java类加载器(ClassLoader)</td><td><a href="https://blog.csdn.net/javazejian/article/details/73413292">https://blog.csdn.net/javazejian/article/details/73413292</a></td></tr><tr><td>https原理通俗了解</td><td><a href="https://www.cnblogs.com/zhangshitong/p/6478721.html" target="_blank" rel="noopener">https://www.cnblogs.com/zhangshitong/p/6478721.html</a></td></tr><tr><td>Kafka 的这些原理你知道吗</td><td><a href="https://segmentfault.com/a/1190000021370626" target="_blank" rel="noopener">https://segmentfault.com/a/1190000021370626</a></td></tr><tr><td>Flink基础知识详解</td><td><a href="http://www.54tianzhisheng.cn/tags/Flink/" target="_blank" rel="noopener">http://www.54tianzhisheng.cn/tags/Flink/</a></td></tr><tr><td>Dubbo常见面试题</td><td><a href="https://segmentfault.com/a/1190000037434049" target="_blank" rel="noopener">https://segmentfault.com/a/1190000037434049</a></td></tr><tr><td>敖丙：知识总结</td><td><a href="https://segmentfault.com/u/aobing/articles?page=1" target="_blank" rel="noopener">https://segmentfault.com/u/aobing/articles?page=1</a></td></tr></tbody></table>]]></content>
<summary type="html">
<p>以下是我平时阅读发现比较有深度的文章；做个记录</p>
<table>
<thead>
<tr>
<th>文章标题</th>
<th>地址</th>
</tr>
</thead>
<tbody>
<tr>
<td>RPC原理</td>
<td><a href="http:/
</summary>
</entry>
<entry>
<title>Java垃圾收集器</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/12/07/Java%E5%9E%83%E5%9C%BE%E6%94%B6%E9%9B%86%E5%99%A8/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/12/07/Java垃圾收集器/</id>
<published>2018-12-07T07:24:16.000Z</published>
<updated>2018-12-07T07:28:03.500Z</updated>
<content type="html"><![CDATA[<h3 id="GC的分类"><a href="#GC的分类" class="headerlink" title="GC的分类"></a>GC的分类</h3><blockquote><p>GC的主要回收区域就是年轻代(young gen)、老年代(tenured gen)、持久区（perm gen）,在jdk8之后，perm gen消失，被替换成了元空间（Metaspace）,元空间会在普通的堆区进行分配。垃圾收集为了提高效率，采用分代收集的方式，对于不同特点的回收区域使用不同的垃圾收集器。系统正常运行情况young是比较频繁的，full gc会触发整个heap的扫描和回收。在G1垃圾收集器中，最好的优化状态就是通过不断调整分区空间，避免进行full gc，可以大幅提高吞吐量。下面会详细介绍。</p></blockquote><h3 id="串行垃圾回收器"><a href="#串行垃圾回收器" class="headerlink" title="串行垃圾回收器"></a>串行垃圾回收器</h3><p>单线程回收，并且会有stop theworld（下文会简称STW），也即GC时，暂停所有用户线程。其运行方式是单线程的，适合Client模式的应用，适合单CPU环境。串行的垃圾收集器有两种，Serial以及SerialOld，一般会搭配使用。新生代使用Serial采取复制算法，老年代使用Serial Old采取标记整理算法。Client应用或者命令行程序可以，通过-XX:+UseSerialGC可以开启上述回收模式。</p><blockquote><p>Serial：用于新生代垃圾收集 (复制算法)</p></blockquote><blockquote><p>SerialOld：用于老年代垃圾收集 (标记整理算法)</p></blockquote><h3 id="并行垃圾回收器"><a href="#并行垃圾回收器" class="headerlink" title="并行垃圾回收器"></a>并行垃圾回收器</h3><p>整体来说，并行垃圾回收相对于串行，是通过多线程运行垃圾收集的。也会stop-the-world。适合Server模式以及多CPU环境。一般会和jdk1.5之后出现的CMS搭配使用。并行的垃圾回收器有以下几种：</p><blockquote><p>ParNew：Serial收集器的多线程版本，默认开启的收集线程数和cpu数量一样，运行数量可以通过修改ParallelGCThreads设定。用于新生代收集，使用-XX:+UseParNewGC,和Serial Old收集器组合进行内存回收。(复制算法)</p></blockquote><blockquote><p>Parallel Scavenge: 关注吞吐量,吞吐量优先，吞吐量=代码运行时间/(代码运行时间+垃圾收集时间),也就是高效率利用cpu时间，尽快完成程序的运算任务<br>可以设置最大停顿时间MaxGCPauseMillis以及，吞吐量大小GCTimeRatio。如果设置了-XX:+UseAdaptiveSizePolicy参数，则随着GC,会动态调整新生代的大小，Eden,Survivor比例等，以提供最合适的停顿时间或者最大的吞吐量。用于新生代收集。通过-XX:+UseParallelGC参数，Server模式下默认提供了其和SerialOld进行搭配的分代收集方式 （复制算法）</p></blockquote><blockquote><p>Parllel Old：Parallel Scavenge的老年代版本。JDK 1.6开始提供的。在此之前Parallel Scavenge的地位也很尴尬，而有了Parllel Old之后，通过-XX:+UseParallelOldGC参数使用Parallel Scavenge + Parallel Old器组合进行内存回收。</p></blockquote><h3 id="并发标记扫描垃圾回收器-CMS"><a href="#并发标记扫描垃圾回收器-CMS" class="headerlink" title="并发标记扫描垃圾回收器(CMS)"></a>并发标记扫描垃圾回收器(CMS)</h3><p>CMS（Concurrent Mark Sweep）基于<code>"标记—清除"</code>算法，用于老年代,所以其关注点在于减少“pause time”也即因垃圾回收导致的stop the world时间。对于重视服务的响应速度的应用可以使用CMS。因为CMS是“并发”运行的，也即垃圾收集线程可以和用户线程同时运行。 缺点就是会产生内存碎片。 -XX:UseConcMarkSweepGC参数可以开启CMS,年轻代使用ParNew，老年代使用CMS，同时Serial Old收集器将作为CMS收集器出现Concurrent Mode Failure失败后的后备收集器使用。<br>CMS的回收分为几个阶段：</p><blockquote><p>初始标记：标记一下GC Roots能直接关联到的对象，会“Stop The World”</p></blockquote><blockquote><p>并发标记：GC Roots Tracing，可以和用户线程并发执行。</p></blockquote><blockquote><p>重新标记：标记期间产生的对象存活的再次判断，修正对这些对象的标记，执行时间相对并发标记短，会“Stop The World”。</p></blockquote><blockquote><p>并发清除：清除对象,可以和用户线程并发执行。</p></blockquote><h3 id="G1垃圾收集器"><a href="#G1垃圾收集器" class="headerlink" title="G1垃圾收集器"></a>G1垃圾收集器</h3><blockquote><p>G1垃圾收集器也是以关注延迟为目标、服务器端应用的垃圾收集器，被HotSpot团队寄予取代CMS的使命，也是一个非常具有调优潜力的垃圾收集器。虽然G1也有类似CMS的收集动作：初始标记、并发标记、重新标记、清除、转移回收，并且也以一个串行收集器做担保机制，但单纯地以类似前三种的过程描述显得并不是很妥当。事实上，G1收集与以上三组收集器有很大不同：</p></blockquote><blockquote><ol><li>G1的设计原则是”首先收集尽可能多的垃圾(Garbage First)”。因此，G1并不会等内存耗尽(串行、并行)或者快耗尽(CMS)的时候开始垃圾收集，而是在内部采用了启发式算法，在老年代找出具有高收集收益的分区进行收集。同时G1可以根据用户设置的暂停时间目标自动调整年轻代和总堆大小，暂停目标越短年轻代空间越小、总空间就越大；</li><li>G1采用内存分区(Region)的思路，将内存划分为一个个相等大小的内存分区，回收时则以分区为单位进行回收，存活的对象复制到另一个空闲分区中。由于都是以相等大小的分区为单位进行操作，因此G1天然就是一种压缩方案(局部压缩)；</li><li>G1虽然也是分代收集器，但整个内存分区不存在物理上的年轻代与老年代的区别，也不需要完全独立的survivor(to space)堆做复制准备。G1只有逻辑上的分代概念，或者说每个分区都可能随G1的运行在不同代之间前后切换；</li><li>G1的收集都是STW的，但年轻代和老年代的收集界限比较模糊，采用了混合(mixed)收集的方式。即每次收集既可能只收集年轻代分区(年轻代收集)，也可能在收集年轻代的同时，包含部分老年代分区(混合收集)，这样即使堆</li></ol></blockquote><h3 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h3><blockquote><p>Serial收集器（复制算法)：新生代单线程收集器，标记和清理都是单线程，优点是简单高效。是client级别默认的GC方式，可以通过-XX:+UseSerialGC来强制指定。</p></blockquote><blockquote><p>Serial Old收集器(标记-整理算法)：老年代单线程收集器，Serial收集器的老年代版本。</p></blockquote><blockquote><p>ParNew收集器(停止-复制算法)：新生代收集器，可以认为是Serial收集器的多线程版本,在多核CPU环境下有着比Serial更好的表现。</p></blockquote><blockquote><p>Parallel Scavenge收集器(停止-复制算法)：并行收集器，追求高吞吐量，高效利用CPU。吞吐量一般为99%， 吞吐量= 用户线程时间/(用户线程时间+GC线程时间)。适合后台应用等对交互相应要求不高的场景。是server级别默认采用的GC方式，可用-XX:+UseParallelGC来强制指定，用-XX:ParallelGCThreads=4来指定线程数。</p></blockquote><blockquote><p>Parallel Old收集器(停止-复制算法)：Parallel Scavenge收集器的老年代版本，并行收集器，吞吐量优先。</p></blockquote><blockquote><p>CMS(Concurrent Mark Sweep)收集器（标记-清理算法）：高并发、低停顿，追求最短GC回收停顿时间，cpu占用比较高，响应时间快，停顿时间短，多核cpu 追求高响应时间的选择。</p></blockquote><blockquote><p>G1可以根据用户设置的暂停时间目标自动调整年轻代和总堆大小，暂停目标越短年轻代空间越小、总空间就越大;当JVM分配对象到Eden区域失败(Eden区已满)时,触发young gc；年轻代收集不断活动后，老年代的空间也会被逐渐填充。当老年代占用空间超过整堆比IHOP阈值-XX:InitiatingHeapOccupancyPercent(默认45%)时，G1就会启动一次混合垃圾收集（）周期</p></blockquote><h5 id="知乎看法一："><a href="#知乎看法一：" class="headerlink" title="知乎看法一："></a>知乎看法一：</h5><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">针对HotSpot VM的实现，它里面的GC其实准确分类只有两大种：</span><br><span class="line">Partial GC：并不收集整个GC堆的模式</span><br><span class="line">Young GC：只收集young gen的GC</span><br><span class="line">Old GC：只收集old gen的GC。只有CMS的concurrentcollection是这个模式</span><br><span class="line">Mixed GC：收集整个young gen以及部分old gen的GC。只有G1有这个模式</span><br><span class="line">Full GC：收集整个堆，包括young gen、old gen、perm gen（如果存在的话）等所有部分的模式。</span><br><span class="line">Major GC通常是跟full GC是等价的，收集整个GC堆。但因为HotSpot VM发展了这么多年，外界对各种名词的解读已经完全混乱了，当有人说“major GC”的时候一定要问清楚他想要指的是上面的full GC还是old GC。</span><br><span class="line"></span><br><span class="line">最简单的分代式GC策略，按HotSpot VM的serial GC的实现来看，触发条件是：</span><br><span class="line">young GC：当young gen中的eden区分配满的时候触发。注意young GC中有部分存活对象会晋升到old gen，所以young GC后old gen的占用量通常会有所升高。</span><br><span class="line">full GC：当准备要触发一次young GC时，如果发现统计数据说之前young GC的平均晋升大小比目前old gen剩余的空间大，则不会触发young GC而是转为触发full GC（因为HotSpot VM的GC里，除了CMS的concurrent collection之外，其它能收集old gen的GC都会同时收集整个GC堆，包括young gen，所以不需要事先触发一次单独的young GC）；或者，如果有perm gen的话，要在perm gen分配空间但已经没有足够空间时，也要触发一次full GC；或者System.gc()、heap dump带GC，默认也是触发full GC。</span><br><span class="line"></span><br><span class="line">HotSpot VM里其它非并发GC的触发条件复杂一些，不过大致的原理与上面说的其实一样。当然也总有例外。Parallel Scavenge（-XX:+UseParallelGC）框架下，默认是在要触发full GC前先执行一次young GC，并且两次GC之间能让应用程序稍微运行一小下，以期降低full GC的暂停时间（因为young GC会尽量清理了young gen的死对象，减少了full GC的工作量）。控制这个行为的VM参数是-XX:+ScavengeBeforeFullGC。这是HotSpot VM里的奇葩嗯。可跳传送门围观：JVM full GC的奇怪现象，求解惑？ - RednaxelaFX 的回答并发GC的触发条件就不太一样。以CMS GC为例，它主要是定时去检查old gen的使用量，当使用量超过了触发比例就会启动一次CMS GC，对old gen做并发收集。</span><br></pre></td></tr></table></figure><h5 id="知乎看法二："><a href="#知乎看法二：" class="headerlink" title="知乎看法二："></a>知乎看法二：</h5><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">1. Full GC定义是相对明确的，就是针对整个新生代、老生代、元空间（metaspace，java8以上版本取代perm gen）的全局范围的GC；</span><br><span class="line">2. Minor GC和Major GC是俗称，在Hotspot JVM实现的Serial GC, Parallel GC, CMS, G1 GC中大致可以对应到某个Young GC和Old GC算法组合；</span><br><span class="line">3. 最重要是搞明白上述Hotspot JVM实现中几种GC算法组合到底包含了什么。</span><br><span class="line">3.1 Serial GC算法：Serial Young GC ＋ Serial Old GC （敲黑板！敲黑板！敲黑板！实际上它是全局范围的Full GC）；</span><br><span class="line">3.2 Parallel GC算法：Parallel Young GC ＋ 非并行的PS MarkSweep GC / 并行的Parallel Old GC（敲黑板！敲黑板！敲黑板！这俩实际上也是全局范围的Full GC），选PS MarkSweep GC 还是 Parallel Old GC 由参数UseParallelOldGC来控制；</span><br><span class="line">3.3 CMS算法：ParNew（Young）GC + CMS（Old）GC （piggyback on ParNew的结果／老生代存活下来的object只做记录，不做compaction）＋ Full GC for CMS算法（应对核心的CMS GC某些时候的不赶趟，开销很大）；</span><br><span class="line">3.4 G1 GC：Young GC + mixed GC（新生代，再加上部分老生代）＋ Full GC for G1 GC算法（应对G1 GC算法某些时候的不赶趟，开销很大）；</span><br><span class="line">4. 搞清楚了上面这些组合，我们再来看看各类GC算法的触发条件。简单说，触发条件就是某GC算法对应区域满了，或是预测快满了。比如，</span><br><span class="line">4.1 各种Young GC的触发原因都是eden区满了；</span><br><span class="line">4.2 Serial Old GC／PS MarkSweep GC／Parallel Old GC的触发则是在要执行Young GC时候预测其promote的object的总size超过老生代剩余size；</span><br><span class="line">4.3 CMS GC的initial marking的触发条件是老生代使用比率超过某值；</span><br><span class="line">4.4 G1 GC的initial marking的触发条件是Heap使用比率超过某值，跟4.3 heuristics 类似；</span><br><span class="line">4.5 Full GC for CMS算法和Full GC for G1 GC算法的触发原因很明显，就是4.3 和 4.4 的fancy算法不赶趟了，只能全局范围大搞一次GC了（相信我，这很慢！这很慢！这很慢！）；</span><br><span class="line">5 题主说的 “Full GC会先触发一次Minor GC” － 指的应该是</span><br><span class="line">5.1 PS MarkSweep GC／Parallel Old GC（Full GC）之前会跑一次Parallel Young GC；</span><br><span class="line">原因就是减轻Full GC 的负担。</span><br><span class="line">哇～整个picture 是有点乱，希望我整理的还算清楚</span><br></pre></td></tr></table></figure><h3 id="参考文章"><a href="#参考文章" class="headerlink" title="参考文章"></a>参考文章</h3><p>垃圾回收器说明：<a href="https://blog.csdn.net/lijingyao8206/article/details/80513383">https://blog.csdn.net/lijingyao8206/article/details/80513383</a></p><p>垃圾回收算法说明：<a href="https://www.cnblogs.com/1024Community/p/honery.html" target="_blank" rel="noopener">https://www.cnblogs.com/1024Community/p/honery.html</a></p><p>知乎大神看法：<a href="https://www.zhihu.com/question/41922036/answer/93079526" target="_blank" rel="noopener">https://www.zhihu.com/question/41922036/answer/93079526</a></p>]]></content>
<summary type="html">
<h3 id="GC的分类"><a href="#GC的分类" class="headerlink" title="GC的分类"></a>GC的分类</h3><blockquote>
<p>GC的主要回收区域就是年轻代(young gen)、老年代(tenured gen)、持久
</summary>
</entry>
<entry>
<title>JVM性能查看分析</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/12/04/JVM%E6%80%A7%E8%83%BD%E6%9F%A5%E7%9C%8B%E5%88%86%E6%9E%90/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/12/04/JVM性能查看分析/</id>
<published>2018-12-04T09:07:37.000Z</published>
<updated>2018-12-04T10:19:05.299Z</updated>
<content type="html"><![CDATA[<h4 id="一、uptime"><a href="#一、uptime" class="headerlink" title="一、uptime"></a>一、uptime</h4><p>查看系统负载</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">[admin@10-57-19-253 ~]$ uptime</span><br><span class="line"> 16:34:16 up 36 days, 5:49, 1 user, load average: 0.00, 0.01, 0.05</span><br></pre></td></tr></table></figure><p>解析：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">16:34:16 //当前系统时间</span><br><span class="line">up 1 day, 5:49 //持续运行时间,时间越大，说明你的机器越稳定。 </span><br><span class="line">1 users //用户连接数，是总连接数而不是用户数 </span><br><span class="line">average: 0.00, 0.01, 0.05 //系统平均负载，统计最近1，5，15分钟的系统平均负载</span><br></pre></td></tr></table></figure><h4 id="二、top"><a href="#二、top" class="headerlink" title="二、top"></a>二、top</h4><p>查看系统PID、进程占用CPU、内存、用户</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br></pre></td><td class="code"><pre><span class="line">top - 16:31:40 up 36 days, 5:47, 1 user, load average: 0.00, 0.01, 0.05</span><br><span class="line">Tasks: 112 total, 1 running, 111 sleeping, 0 stopped, 0 zombie</span><br><span class="line">%Cpu(s): 0.0 us, 0.1 sy, 0.0 ni, 99.9 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st</span><br><span class="line">KiB Mem : 8010176 total, 6052396 free, 1054808 used, 902972 buff/cache</span><br><span class="line">KiB Swap: 0 total, 0 free, 0 used. 6621668 avail Mem</span><br><span class="line"></span><br><span class="line"> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND</span><br><span class="line"> 291 root 20 0 0 0 0 S 0.3 0.0 9:19.76 xfsaild/vda2</span><br><span class="line">18591 admin 20 0 5069284 871172 13056 S 0.3 10.9 214:37.38 java</span><br><span class="line"> 1 root 20 0 128168 6744 3984 S 0.0 0.1 1:00.66 systemd</span><br><span class="line"> 2 root 20 0 0 0 0 S 0.0 0.0 0:00.74 kthreadd</span><br><span class="line"> 3 root 20 0 0 0 0 S 0.0 0.0 0:00.12 ksoftirqd/0</span><br><span class="line"> 5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0H</span><br><span class="line"> 6 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/u8:0</span><br><span class="line"> 7 root rt 0 0 0 0 S 0.0 0.0 0:00.48 migration/0</span><br><span class="line"> 8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh</span><br><span class="line"> 9 root 20 0 0 0 0 S 0.0 0.0 2:37.71 rcu_sched</span><br><span class="line"> 10 root rt 0 0 0 0 S 0.0 0.0 0:09.80 watchdog/0</span><br><span class="line"> 11 root rt 0 0 0 0 S 0.0 0.0 0:08.44 watchdog/1</span><br><span class="line"> 12 root rt 0 0 0 0 S 0.0 0.0 0:00.34 migration/1</span><br><span class="line"> 13 root 20 0 0 0 0 S 0.0 0.0 0:01.15 ksoftirqd/1</span><br><span class="line"> 15 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/1:0H</span><br><span class="line"> 16 root rt 0 0 0 0 S 0.0 0.0 0:08.99 watchdog/2</span><br><span class="line"> 17 root rt 0 0 0 0 S 0.0 0.0 0:00.52 migration/2</span><br><span class="line"> 18 root 20 0 0 0 0 S 0.0 0.0 0:00.13 ksoftirqd/2</span><br></pre></td></tr></table></figure><p>可知java进程<code>PID</code>为<code>18591</code>.</p><h5 id="top-p-PID"><a href="#top-p-PID" class="headerlink" title="top -p PID"></a><code>top -p PID</code></h5><p>查看指定java进程的CPU/内存使用情况。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">[admin@10-57-19-253 ~]$ top -p 18591</span><br><span class="line">top - 16:35:58 up 36 days, 5:51, 1 user, load average: 0.00, 0.01, 0.05</span><br><span class="line">Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie</span><br><span class="line">%Cpu(s): 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st</span><br><span class="line">KiB Mem : 8010176 total, 6042444 free, 1055312 used, 912420 buff/cache</span><br><span class="line">KiB Swap: 0 total, 0 free, 0 used. 6621028 avail Mem</span><br><span class="line"></span><br><span class="line"> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND</span><br><span class="line">18591 admin 20 0 5069284 871172 13056 S 0.0 10.9 214:38.43 java</span><br></pre></td></tr></table></figure><h5 id="top-Hp-PID"><a href="#top-Hp-PID" class="headerlink" title="top -Hp PID"></a><code>top -Hp PID</code></h5><p>根据进程查找内部线程内存和CPU使用情况</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br></pre></td><td class="code"><pre><span class="line">admin@10-57-19-253 ~]$ top -Hp 18591</span><br><span class="line">top - 16:53:32 up 36 days, 6:09, 1 user, load average: 0.00, 0.01, 0.05</span><br><span class="line">Threads: 334 total, 0 running, 334 sleeping, 0 stopped, 0 zombie</span><br><span class="line">%Cpu(s): 0.1 us, 0.0 sy, 0.0 ni, 99.9 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st</span><br><span class="line">KiB Mem : 8010176 total, 5946812 free, 1051908 used, 1011456 buff/cache</span><br><span class="line">KiB Swap: 0 total, 0 free, 0 used. 6622372 avail Mem</span><br><span class="line"></span><br><span class="line"> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND</span><br><span class="line">18677 admin 20 0 5069284 875844 13088 S 0.3 10.9 56:00.08 java</span><br><span class="line">18591 admin 20 0 5069284 875844 13088 S 0.0 10.9 0:00.00 java</span><br><span class="line">18592 admin 20 0 5069284 875844 13088 S 0.0 10.9 0:00.58 java</span><br><span class="line">18593 admin 20 0 5069284 875844 13088 S 0.0 10.9 0:02.83 java</span><br><span class="line">18594 admin 20 0 5069284 875844 13088 S 0.0 10.9 0:02.89 java</span><br><span class="line">18595 admin 20 0 5069284 875844 13088 S 0.0 10.9 0:02.95 java</span><br><span class="line">18596 admin 20 0 5069284 875844 13088 S 0.0 10.9 0:02.98 java</span><br><span class="line">18597 admin 20 0 5069284 875844 13088 S 0.0 10.9 2:55.02 java</span><br><span class="line">18598 admin 20 0 5069284 875844 13088 S 0.0 10.9 0:00.25 java</span><br><span class="line">18599 admin 20 0 5069284 875844 13088 S 0.0 10.9 0:00.25 java</span><br><span class="line">18600 admin 20 0 5069284 875844 13088 S 0.0 10.9 0:00.00 java</span><br><span class="line">18601 admin 20 0 5069284 875844 13088 S 0.0 10.9 0:00.00 java</span><br><span class="line">18602 admin 20 0 5069284 875844 13088 S 0.0 10.9 0:00.00 java</span><br><span class="line">18603 admin 20 0 5069284 875844 13088 S 0.0 10.9 0:55.14 java</span><br><span class="line">18604 admin 20 0 5069284 875844 13088 S 0.0 10.9 0:51.45 java</span><br><span class="line">18605 admin 20 0 5069284 875844 13088 S 0.0 10.9 0:22.95 java</span><br><span class="line">18606 admin 20 0 5069284 875844 13088 S 0.0 10.9 0:00.00 java</span><br><span class="line">18607 admin 20 0 5069284 875844 13088 S 0.0 10.9 19:09.66 java</span><br></pre></td></tr></table></figure><h4 id="三、jps"><a href="#三、jps" class="headerlink" title="三、jps"></a>三、jps</h4><p>查看java进程PID</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">[admin@10-57-19-253 ~]$ jps</span><br><span class="line">26668 Jps</span><br><span class="line">18591 Bootstrap</span><br></pre></td></tr></table></figure><h4 id="四、jstat"><a href="#四、jstat" class="headerlink" title="四、jstat"></a>四、jstat</h4><p>用于输出java程序内存使用情况，包括新生代、老年代、元数据区容量、垃圾回收情况; 命令<code>jstat -gcutil PID interval count</code>：表示没1000毫秒统计一次结果，共5次</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">[admin@10-57-19-253 ~]$ jstat -gcutil 18591 1000 5</span><br><span class="line"> S0 S1 E O M CCS YGC YGCT FGC FGCT GCT</span><br><span class="line"> 0.00 67.17 27.74 17.08 97.29 94.54 1201 6.722 3 0.280 7.003</span><br><span class="line"> 0.00 67.17 27.74 17.08 97.29 94.54 1201 6.722 3 0.280 7.003</span><br><span class="line"> 0.00 67.17 27.74 17.08 97.29 94.54 1201 6.722 3 0.280 7.003</span><br><span class="line"> 0.00 67.17 28.55 17.08 97.29 94.54 1201 6.722 3 0.280 7.003</span><br><span class="line"> 0.00 67.17 28.55 17.08 97.29 94.54 1201 6.722 3 0.280 7.003</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">S0：幸存1区当前使用比例</span><br><span class="line">S1：幸存2区当前使用比例</span><br><span class="line">E：伊甸园区使用比例</span><br><span class="line">O：老年代使用比例</span><br><span class="line">M：元数据区使用比例</span><br><span class="line">CCS：压缩使用比例</span><br><span class="line">YGC：年轻代垃圾回收次数</span><br><span class="line">FGC：老年代垃圾回收次数</span><br><span class="line">FGCT：老年代垃圾回收消耗时间</span><br><span class="line">GCT：垃圾回收消耗总时间</span><br></pre></td></tr></table></figure><h4 id="五、jmap"><a href="#五、jmap" class="headerlink" title="五、jmap"></a>五、jmap</h4><p>用于输出java程序中内存对象的情况，包括有哪些对象，对象的数量。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br></pre></td><td class="code"><pre><span class="line">jmap -histo 18591</span><br><span class="line"> num #instances #bytes class name</span><br><span class="line">----------------------------------------------</span><br><span class="line"> 1: 631652 64523944 [C</span><br><span class="line"> 2: 179202 57308168 [B</span><br><span class="line"> 3: 32920 23425720 [I</span><br><span class="line"> 4: 587502 14100048 java.lang.String</span><br><span class="line"> 5: 346589 11090848 java.util.ArrayList$Itr</span><br><span class="line"> 6: 135337 5917832 [Ljava.lang.Object;</span><br><span class="line"> 7: 168929 5405728 java.util.HashMap$Node</span><br><span class="line"> 8: 81666 4573296 java.util.concurrent.ConcurrentHashMap$ValueIterator</span><br><span class="line"> 9: 76926 4307856 java.util.concurrent.ConcurrentHashMap$KeyIterator</span><br><span class="line"> 10: 107003 4280120 java.util.HashMap$EntryIterator</span><br><span class="line"> 11: 70716 3394368 java.util.HashMap</span><br><span class="line"> 12: 140482 3371568 java.util.ArrayList</span><br><span class="line"> 13: 55494 3107664 java.util.LinkedHashMap</span><br><span class="line"> 14: 34441 3030808 java.lang.reflect.Method</span><br><span class="line"> 15: 26545 2785344 [Ljava.util.HashMap$Node;</span><br><span class="line"> 16: 77308 2473856 java.util.LinkedHashMap$LinkedEntryIterator</span><br><span class="line"> 17: 66838 2138816 java.util.concurrent.ConcurrentHashMap$Node</span><br><span class="line"> 18: 68194 1636656 org.apache.kafka.common.internals.PartitionStates$PartitionState</span><br><span class="line"> 19: 40476 1619040 java.util.HashMap$KeyIterator</span><br><span class="line"> 20: 39732 1589280 java.util.LinkedHashMap$Entry</span><br><span class="line"> 21: 13979 1557928 java.lang.Class</span><br><span class="line"> 22: 13926 1225488 org.apache.naming.resources.FileDirContext$FileResourceAttributes</span><br><span class="line"> 23: 29725 951200 java.util.LinkedList</span><br><span class="line"> 24: 14451 924864 java.net.URL</span><br><span class="line"> 25: 26984 863488 java.io.File</span><br><span class="line"> ......</span><br><span class="line"> ......</span><br><span class="line"> Total 3984505 254742192</span><br></pre></td></tr></table></figure><p>上述命令打印出进程ID为18591的内存情况。但我们常用的方式是将指定进程的内存heap输出到外部文件，再由专门的heap分析工具进行分析,例如mat（Memory Analysis Tool），所以我们常用的命令是：</p><p>jmap -dump:live,format=b,file=heap.hprof 18591</p><h4 id="六、jstack"><a href="#六、jstack" class="headerlink" title="六、jstack"></a>六、jstack</h4><p>导出线程栈</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">jstack 18591 > jstack.out</span><br></pre></td></tr></table></figure><p>运行结果：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br></pre></td><td class="code"><pre><span class="line">2018-12-04 16:48:47</span><br><span class="line">Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.60-b23 mixed mode):</span><br><span class="line"></span><br><span class="line">"Attach Listener" #59821 daemon prio=9 os_prio=0 tid=0x00007f3118047800 nid=0x68ad waiting on condition [0x0000000000000000]</span><br><span class="line"> java.lang.Thread.State: RUNNABLE</span><br><span class="line"></span><br><span class="line">"pool-3-thread-3" #25984 prio=5 os_prio=0 tid=0x00007f30c4125800 nid=0x64ee waiting on condition [0x00007f30a2eae000]</span><br><span class="line"> java.lang.Thread.State: WAITING (parking)</span><br><span class="line"> at sun.misc.Unsafe.park(Native Method)</span><br><span class="line"> - parking to wait for <0x00000000c2b1cd90> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)</span><br><span class="line"> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)</span><br><span class="line"> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)</span><br><span class="line"> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)</span><br><span class="line"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)</span><br><span class="line"> at java.lang.Thread.run(Thread.java:745)</span><br><span class="line"></span><br><span class="line">"KPOLLER-CC-topic" #77 daemon prio=5 os_prio=0 tid=0x00007f30e1839000 nid=0x48f5 runnable [0x00007f30b7efd000]</span><br><span class="line"> java.lang.Thread.State: RUNNABLE</span><br><span class="line"> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)</span><br><span class="line"> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)</span><br><span class="line"> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)</span><br><span class="line"> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)</span><br><span class="line"> - locked <0x00000000c2ddb3b0> (a sun.nio.ch.Util$2)</span><br><span class="line"> - locked <0x00000000c2ddb3a0> (a java.util.Collections$UnmodifiableSet)</span><br><span class="line"> - locked <0x00000000c2ddb0d0> (a sun.nio.ch.EPollSelectorImpl)</span><br><span class="line"> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)</span><br><span class="line"> at org.apache.kafka.common.network.Selector.select(Selector.java:684)</span><br><span class="line"> at org.apache.kafka.common.network.Selector.poll(Selector.java:408)</span><br><span class="line"> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:460)</span><br><span class="line"> at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:261)</span><br><span class="line"> at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233)</span><br><span class="line"> at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1171)</span><br><span class="line"> at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1115)</span><br><span class="line">"main" #1 prio=5 os_prio=0 tid=0x00007f315000c000 nid=0x48a0 runnable [0x00007f3157be7000]</span><br><span class="line"> java.lang.Thread.State: RUNNABLE</span><br><span class="line">at java.net.PlainSocketImpl.socketAccept(Native Method)</span><br><span class="line">at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)</span><br><span class="line">at java.net.ServerSocket.implAccept(ServerSocket.java:545)</span><br><span class="line">at java.net.ServerSocket.accept(ServerSocket.java:513)</span><br><span class="line">at org.apache.catalina.core.StandardServer.await(StandardServer.java:470)</span><br><span class="line">at org.apache.catalina.startup.Catalina.await(Catalina.java:781)</span><br><span class="line">at org.apache.catalina.startup.Catalina.start(Catalina.java:727)</span><br><span class="line">at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)</span><br><span class="line">at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)</span><br><span class="line">at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)</span><br><span class="line">at java.lang.reflect.Method.invoke(Method.java:497)</span><br><span class="line">at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:294)</span><br><span class="line">at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:428)</span><br><span class="line"></span><br><span class="line">"VM Thread" os_prio=0 tid=0x00007f3150079800 nid=0x48a5 runnable</span><br><span class="line"></span><br><span class="line">"GC task thread#0 (ParallelGC)" os_prio=0 tid=0x00007f3150021000 nid=0x48a1 runnable</span><br><span class="line"></span><br><span class="line">"GC task thread#1 (ParallelGC)" os_prio=0 tid=0x00007f3150023000 nid=0x48a2 runnable</span><br><span class="line"></span><br><span class="line">"GC task thread#2 (ParallelGC)" os_prio=0 tid=0x00007f3150024800 nid=0x48a3 runnable</span><br><span class="line"></span><br><span class="line">"GC task thread#3 (ParallelGC)" os_prio=0 tid=0x00007f3150026800 nid=0x48a4 runnable</span><br><span class="line"></span><br><span class="line">"VM Periodic Task Thread" os_prio=0 tid=0x00007f31500d7800 nid=0x48af waiting on condition</span><br><span class="line"></span><br><span class="line">JNI global references: 11741</span><br></pre></td></tr></table></figure><h4 id="七、printf将线程tid转换为16进制"><a href="#七、printf将线程tid转换为16进制" class="headerlink" title="七、printf将线程tid转换为16进制"></a>七、printf将线程tid转换为16进制</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">[admin@10-57-19-253 ~]$ printf '%x\n' 18677</span><br><span class="line">48f5</span><br></pre></td></tr></table></figure><h4 id="八、vmstat"><a href="#八、vmstat" class="headerlink" title="八、vmstat"></a>八、vmstat</h4><p>查看系统CPU/内存、swap、io等情况 ; <code>vmstat interval count</code>：每隔1秒输出一个结果，共5次</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">[admin@10-57-19-253 ~]$ vmstat 1 5</span><br><span class="line">procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----</span><br><span class="line"> r b swpd free buff cache si so bi bo in cs us sy id wa st</span><br><span class="line"> 1 0 0 5941272 2068 1009732 0 0 0 0 0 0 0 0 100 0 0</span><br><span class="line"> 0 0 0 5941272 2068 1009732 0 0 0 12 392 394 0 0 100 0 0</span><br><span class="line"> 0 0 0 5941272 2068 1009732 0 0 0 54 410 440 0 0 100 0 0</span><br><span class="line"> 0 0 0 5941272 2068 1009732 0 0 0 0 363 381 0 0 100 0 0</span><br><span class="line"> 0 0 0 5941272 2068 1009732 0 0 0 0 358 384 0 0 100 0 0</span><br></pre></td></tr></table></figure><p>解析：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line">进程</span><br><span class="line">r: 运行队列中进程数量，这个值也可以判断是否需要增加CPU。（长期大于1）</span><br><span class="line">b: 等待IO的进程数量。</span><br><span class="line"></span><br><span class="line">内存</span><br><span class="line">swpd: 使用虚拟内存大小，如果swpd的值不为0，但是SI，SO的值长期为0，这种情况不会影响系统性能。</span><br><span class="line">free: 空闲物理内存大小。</span><br><span class="line">buff: 用作缓冲的内存大小。</span><br><span class="line">cache: 用作缓存的内存大小，如果cache的值大的时候，说明cache处的文件数多，如果频繁访问到的文件都能被cache处，那么磁盘的读IO bi会非常小。</span><br><span class="line"></span><br><span class="line">Swap</span><br><span class="line">si:每秒从交换区写到内存的大小，由磁盘调入内存。</span><br><span class="line">so:每秒写入交换区的内存大小，由内存调入磁盘。</span><br><span class="line"></span><br><span class="line">CPU</span><br><span class="line">us:用户进程执行时间百分比(user time) us的值比较高时，说明用户进程消耗的CPU时间多，但是如果长期超50%的使用，那么我们就该考虑优化程序算法或者进行加速。</span><br><span class="line">sy:内核系统进程执行时间百分比(system time) sy的值高时，说明系统内核消耗的CPU资源多，这并不是良性表现，我们应该检查原因。</span><br><span class="line">wa:IO等待时间百分比 wa的值高时，说明IO等待比较严重，这可能由于磁盘大量作随机访问造成，也有可能磁盘出现瓶颈（块操作）。</span><br><span class="line">id: 空闲时间百分比</span><br></pre></td></tr></table></figure><h4 id="九、应用"><a href="#九、应用" class="headerlink" title="九、应用"></a>九、应用</h4><p>查看哪个线程占用内存最高问题</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">1) `top` 查看内存和CPU使用最高的进程PID</span><br><span class="line">2）`jstack PID > jstack.out` 导出线程栈</span><br><span class="line">3) `top -Hp PID` 查看该进程内各线程使用内存和CPU情况,和进程tid</span><br><span class="line">4）`printf '%x\n' tid`将进程`tid`转换为16进制</span><br><span class="line">5）根据4步骤输出结果去2步骤输出结果`jstack.out`里面检索</span><br></pre></td></tr></table></figure><h4 id="十、参考地址"><a href="#十、参考地址" class="headerlink" title="十、参考地址"></a>十、参考地址</h4><p><a href="https://www.cnblogs.com/leefreeman/p/7464179.html" target="_blank" rel="noopener">https://www.cnblogs.com/leefreeman/p/7464179.html</a></p><p><a href="https://www.cnblogs.com/xqzt/p/5448983.html" target="_blank" rel="noopener">https://www.cnblogs.com/xqzt/p/5448983.html</a></p><p><a href="https://github.com/oldratlee/useful-scripts" target="_blank" rel="noopener">https://github.com/oldratlee/useful-scripts</a></p>]]></content>
<summary type="html">
<h4 id="一、uptime"><a href="#一、uptime" class="headerlink" title="一、uptime"></a>一、uptime</h4><p>查看系统负载</p>
<figure class="highlight plain"><ta
</summary>
<category term="jvm" scheme="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/tags/jvm/"/>
<category term="top" scheme="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/tags/top/"/>
<category term="jstat" scheme="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/tags/jstat/"/>
<category term="jmap" scheme="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/tags/jmap/"/>
<category term="jstack" scheme="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/tags/jstack/"/>
</entry>
<entry>
<title>Java JWT 生成token</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/11/29/Java-JWT-%E7%94%9F%E6%88%90token/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/11/29/Java-JWT-生成token/</id>
<published>2018-11-29T09:34:18.000Z</published>
<updated>2018-11-29T09:35:10.525Z</updated>
<content type="html"><![CDATA[<h3 id="Java-JWT-生成token"><a href="#Java-JWT-生成token" class="headerlink" title="Java JWT 生成token"></a>Java JWT 生成token</h3><blockquote><p>JSON Web Token for Java and Android</p></blockquote><h4 id="Maven依赖"><a href="#Maven依赖" class="headerlink" title="Maven依赖"></a>Maven依赖</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"><dependency></span><br><span class="line"> <groupId>io.jsonwebtoken</groupId></span><br><span class="line"> <artifactId>jjwt-api</artifactId></span><br><span class="line"> <version>0.10.5</version></span><br><span class="line"></dependency></span><br><span class="line"><dependency></span><br><span class="line"> <groupId>io.jsonwebtoken</groupId></span><br><span class="line"> <artifactId>jjwt-impl</artifactId></span><br><span class="line"> <version>0.10.5</version></span><br><span class="line"> <scope>runtime</scope></span><br><span class="line"></dependency></span><br><span class="line"><dependency></span><br><span class="line"> <groupId>io.jsonwebtoken</groupId></span><br><span class="line"> <artifactId>jjwt-jackson</artifactId></span><br><span class="line"> <version>0.10.5</version></span><br><span class="line"> <scope>runtime</scope></span><br><span class="line"></dependency></span><br></pre></td></tr></table></figure><h4 id="JWT参数说明"><a href="#JWT参数说明" class="headerlink" title="JWT参数说明"></a>JWT参数说明</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">iss: jwt签发者</span><br><span class="line">sub: jwt所面向的用户</span><br><span class="line">aud: 接收jwt的一方</span><br><span class="line">exp: jwt的过期时间，这个过期时间必须要大于签发时间</span><br><span class="line">nbf: 定义在什么时间之前，该jwt都是不可用的.</span><br><span class="line">iat: jwt的签发时间</span><br><span class="line">jti: jwt的唯一身份标识，主要用来作为一次性token,从而回避重放攻击。</span><br></pre></td></tr></table></figure><h4 id="JWT-简单代码实现"><a href="#JWT-简单代码实现" class="headerlink" title="JWT 简单代码实现"></a>JWT 简单代码实现</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br></pre></td><td class="code"><pre><span class="line">package jwt;</span><br><span class="line"></span><br><span class="line">import com.alibaba.fastjson.JSON;</span><br><span class="line">import io.jsonwebtoken.*;</span><br><span class="line">import io.jsonwebtoken.security.Keys;</span><br><span class="line"></span><br><span class="line">import java.security.Key;</span><br><span class="line">import java.util.Date;</span><br><span class="line">import java.util.UUID;</span><br><span class="line"></span><br><span class="line">/**</span><br><span class="line"> * <p>JWT 生成token</p></span><br><span class="line"> *</span><br><span class="line"> * @author: zhengyong Date: 2018/11/29 Time: 下午4:48</span><br><span class="line"> */</span><br><span class="line">public class JwtTest {</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * 利用JWT生成token</span><br><span class="line"> *</span><br><span class="line"> * @return token</span><br><span class="line"> */</span><br><span class="line"> public static String generateToken(Key key) {</span><br><span class="line"> /**</span><br><span class="line"> // 以下算法过程是注释伪代码步骤</span><br><span class="line"> String header = "{\"alg\":\"HS256\"}";</span><br><span class="line"> String body = "{\"sub\":\"Joe\"}";</span><br><span class="line"> String encodedHeader = base64URLEncode( header.getBytes("UTF-8"));</span><br><span class="line"> String encodedClaims = base64URLEncode( claims.getBytes("UTF-8") );</span><br><span class="line"> String concatenated = encodedHeader + "." + encodedClaims;</span><br><span class="line"> Key key = getMySecretKey();</span><br><span class="line"> byte[] signature = hmacSha256( concatenated, key );</span><br><span class="line"> String jws = concatenated + '.' + base64URLEncode( signature );</span><br><span class="line"> **/</span><br><span class="line"> // We need a signing key, so we'll create one just for this example. Usually</span><br><span class="line"> // the key would be read from your application configuration instead.</span><br><span class="line"> String jws = Jwts.builder()</span><br><span class="line"> .setIssuer("me")</span><br><span class="line"> .setSubject("Bob")</span><br><span class="line"> .setAudience("you")</span><br><span class="line">// .setExpiration(expiration) //a java.util.Date</span><br><span class="line">// .setNotBefore(notBefore) //a java.util.Date</span><br><span class="line"> .setIssuedAt(new Date()) // for example, now</span><br><span class="line"> .setId(UUID.randomUUID().toString()).signWith(key).compact(); //just an example id</span><br><span class="line"> return jws;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * 验证token</span><br><span class="line"> */</span><br><span class="line"> public static Jws validateToken(String jwsToken, Key key) {</span><br><span class="line"> try {</span><br><span class="line"></span><br><span class="line"> Jws<Claims> claimsJws = Jwts.parser().setSigningKey(key).parseClaimsJws(jwsToken);</span><br><span class="line"> //OK, we can trust this JWT</span><br><span class="line"> return claimsJws;</span><br><span class="line"> } catch (JwtException e) {</span><br><span class="line"> //don't trust the JWT!</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> }</span><br><span class="line"> System.out.println("error");</span><br><span class="line"> return null;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public static void main(String[] args) {</span><br><span class="line"> Key key = Keys.secretKeyFor(SignatureAlgorithm.HS256);</span><br><span class="line"> // 生成token</span><br><span class="line"> String token = generateToken(key);</span><br><span class="line"> System.out.println("token:" + token);</span><br><span class="line"></span><br><span class="line"> // 解析验证token</span><br><span class="line"> Jws jws = validateToken(token, key);</span><br><span class="line"> System.out.println(JSON.toJSONString(jws));</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="参考地址"><a href="#参考地址" class="headerlink" title="参考地址"></a>参考地址</h3><p><a href="https://github.com/jwtk/jjwt" target="_blank" rel="noopener">https://github.com/jwtk/jjwt</a></p>]]></content>
<summary type="html">
<h3 id="Java-JWT-生成token"><a href="#Java-JWT-生成token" class="headerlink" title="Java JWT 生成token"></a>Java JWT 生成token</h3><blockquote>
<p>J
</summary>
</entry>
<entry>
<title>spring cache基于redis实现</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/11/26/spring-cache%E5%9F%BA%E4%BA%8Eredis%E5%AE%9E%E7%8E%B0/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/11/26/spring-cache基于redis实现/</id>
<published>2018-11-26T07:40:37.000Z</published>
<updated>2018-11-26T08:26:27.781Z</updated>
<content type="html"><![CDATA[<h3 id="一、概述"><a href="#一、概述" class="headerlink" title="一、概述"></a>一、概述</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">Spring 3.1 引入了激动人心的基于凝视（annotation）的缓存（cache）技术，它本质上不是一个具体的缓存实现方案（比如EHCache 或者 OSCache），而是一个对缓存使用的抽象，通过在既有代码中加入少量它定义的各种 annotation，即能够达到缓存方法的返回对象的效果。</span><br><span class="line"></span><br><span class="line">Spring 的缓存技术还具备相当的灵活性。不仅能够使用 SpEL（Spring Expression Language）来定义缓存的 key 和各种 condition，还提供开箱即用的缓存暂时存储方案，也支持和主流的专业缓存比如 EHCache 集成。</span><br><span class="line"></span><br><span class="line">其特点总结例如以下：</span><br><span class="line"></span><br><span class="line">通过少量的配置 annotation 凝视就可以使得既有代码支持缓存</span><br><span class="line">支持开箱即用 Out-Of-The-Box，即不用安装和部署额外第三方组件就可以使用缓存</span><br><span class="line">支持 Spring Express Language，能使用对象的不论什么属性或者方法来定义缓存的 key 和 condition</span><br><span class="line">支持 AspectJ，并通过事实上现不论什么方法的缓存支持</span><br><span class="line">支持自己定义 key 和自己定义缓存管理者，具有相当的灵活性和扩展性</span><br><span class="line">本文将针对上述特点对 Spring cache 进行具体的介绍，主要通过一个简单的样例和原理介绍展开，然后我们将一起看一个比較实际的缓存样例。最后会介绍 spring cache 的使用限制和注意事项。</span><br></pre></td></tr></table></figure><h3 id="二、曾经我们如何实现缓存"><a href="#二、曾经我们如何实现缓存" class="headerlink" title="二、曾经我们如何实现缓存"></a>二、曾经我们如何实现缓存</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">public static void main(String[] args) {</span><br><span class="line">// 1. 先查询缓存数据</span><br><span class="line">String cacheData = redisClient.get("key");</span><br><span class="line">// 2. 缓存数据不存在，查询数据库，查询到数据再放入缓存</span><br><span class="line">if(cacheData == null) {</span><br><span class="line">cacheData = loadFromDB("key");</span><br><span class="line">redisClient.put("key", cacheData);</span><br><span class="line">}</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">public static String loadFromDB(String key) {</span><br><span class="line">System.out.println("query data from db, key=" + key);</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="三、利用spring-Cache如何实现缓存"><a href="#三、利用spring-Cache如何实现缓存" class="headerlink" title="三、利用spring Cache如何实现缓存"></a>三、利用spring Cache如何实现缓存</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">public static void main(String[] args) {</span><br><span class="line">// 1. 一句话实现自动保存缓存</span><br><span class="line">String cacheData = cacheData = loadFromDB("key");</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">@Cacheable(value = "myCache", key = "#p0")</span><br><span class="line">public static String loadFromDB(String key) {</span><br><span class="line">System.out.println("query data from db, key=" + key);</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="四、spring-Cache实现简单示列-ConcurrentMap"><a href="#四、spring-Cache实现简单示列-ConcurrentMap" class="headerlink" title="四、spring Cache实现简单示列(ConcurrentMap)"></a>四、spring Cache实现简单示列(ConcurrentMap)</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><dependency></span><br><span class="line"> <groupId>org.springframework</groupId></span><br><span class="line"> <artifactId>spring-context</artifactId></span><br><span class="line"> <version>3.2.4.RELEASE</version></span><br><span class="line"></dependency></span><br></pre></td></tr></table></figure><h5 id="1）XML定义"><a href="#1）XML定义" class="headerlink" title="1）XML定义"></a>1）XML定义</h5><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line"><?xml version="1.0" encoding="UTF-8"?></span><br><span class="line"><beans xmlns="http://www.springframework.org/schema/beans"</span><br><span class="line"> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"</span><br><span class="line"> xmlns:cache="http://www.springframework.org/schema/cache" xmlns:p="http://www.springframework.org/schema/p"</span><br><span class="line"> xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/cache http://www.springframework.org/schema/cache/spring-cache.xsd"></span><br><span class="line"></span><br><span class="line"> <!-- spring cache config--></span><br><span class="line"> <cache:annotation-driven/></span><br><span class="line"> <bean id="userService" class="springcache.UserService"/></span><br><span class="line"> <bean id="redisCustomCache" class="springcache.RedisCustomCache"/></span><br><span class="line"> <bean id="cacheManager" class="org.springframework.cache.support.SimpleCacheManager"></span><br><span class="line"> <property name="caches"></span><br><span class="line"> <set></span><br><span class="line"> <bean class="org.springframework.cache.concurrent.ConcurrentMapCacheFactoryBean" p:name="default"></bean></span><br><span class="line"> <bean class="org.springframework.cache.concurrent.ConcurrentMapCacheFactoryBean" p:name="userCache"></bean></span><br><span class="line"> </set></span><br><span class="line"> </property></span><br><span class="line"> </bean></span><br><span class="line"></span><br><span class="line"></beans></span><br></pre></td></tr></table></figure><h5 id="2）缓存方法实现"><a href="#2）缓存方法实现" class="headerlink" title="2）缓存方法实现"></a>2）缓存方法实现</h5><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br></pre></td><td class="code"><pre><span class="line">package springcache;</span><br><span class="line"></span><br><span class="line">import org.springframework.cache.annotation.CacheEvict;</span><br><span class="line">import org.springframework.cache.annotation.CachePut;</span><br><span class="line">import org.springframework.cache.annotation.Cacheable;</span><br><span class="line"></span><br><span class="line">/**</span><br><span class="line"> * @author: zhengyong Date: 2018/11/23 Time: 上午10:51</span><br><span class="line"> */</span><br><span class="line">public class UserService {</span><br><span class="line"></span><br><span class="line"> // key="#p0" (https://stackoverflow.com/questions/29862053/spring-cache-null-key-returned-for-cache-operation)</span><br><span class="line"> @Cacheable(value = "userCache", key = "#p0")</span><br><span class="line"> public String getUserName(String id) {</span><br><span class="line"> return getFromDB(id);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> @CachePut(value = "userCache", key = "#p0")</span><br><span class="line"> public String updateUser(String id, String name) {</span><br><span class="line"> return getFromDB(id);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> @CacheEvict(value = "userCache", key = "#p0")</span><br><span class="line"> public void deleteUser(String id) {</span><br><span class="line"> System.out.println("delete username from db, id=" + id);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> private String getFromDB(String id) {</span><br><span class="line"> System.out.println("query username from db, id=" + id);</span><br><span class="line"> return String.format("username-%s", id);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h5 id="3）测试类"><a href="#3）测试类" class="headerlink" title="3）测试类"></a>3）测试类</h5><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br></pre></td><td class="code"><pre><span class="line">package springcache;</span><br><span class="line"></span><br><span class="line">import org.apache.log4j.xml.DOMConfigurator;</span><br><span class="line">import org.springframework.context.ApplicationContext;</span><br><span class="line">import org.springframework.context.support.ClassPathXmlApplicationContext;</span><br><span class="line"></span><br><span class="line">/**</span><br><span class="line"> * <p>https://www.ibm.com/developerworks/cn/opensource/os-cn-spring-cache/index.html</p></span><br><span class="line"> * <p></span><br><span class="line"> * Created by zhengyong on 17/3/3.</span><br><span class="line"> */</span><br><span class="line">public class SpringCacheTest {</span><br><span class="line"></span><br><span class="line"> public static void main(String[] args) {</span><br><span class="line"></span><br><span class="line"> UserService userService = (UserService) ctx.getBean("userService");</span><br><span class="line"></span><br><span class="line"> System.out.println(" ---- @Cacheable test ----");</span><br><span class="line"> userService.getUserName("st1");</span><br><span class="line"> userService.getUserName("st1");</span><br><span class="line"> userService.getUserName("st1");</span><br><span class="line"> userService.getUserName("st2");</span><br><span class="line"> userService.getUserName("st3");</span><br><span class="line"> userService.getUserName("st4");</span><br><span class="line"> userService.getUserName("st4");</span><br><span class="line"></span><br><span class="line"> System.out.println(" \n \n");</span><br><span class="line"></span><br><span class="line"> System.out.println(" ---- @CachePut test ----");</span><br><span class="line"> userService.updateUser("st1", null);</span><br><span class="line"> userService.updateUser("st1", null);</span><br><span class="line"> userService.updateUser("st1", null);</span><br><span class="line"> userService.updateUser("st2", null);</span><br><span class="line"> userService.updateUser("st3", null);</span><br><span class="line"> userService.updateUser("st4", null);</span><br><span class="line"> userService.updateUser("st4", null);</span><br><span class="line"></span><br><span class="line"> System.out.println(" \n \n");</span><br><span class="line"></span><br><span class="line"> System.out.println(" ---- @CacheEvict test ----");</span><br><span class="line"> userService.getUserName("st1");</span><br><span class="line"> userService.deleteUser("st1");</span><br><span class="line"> userService.getUserName("st1");</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h5 id="4）运行结果"><a href="#4）运行结果" class="headerlink" title="4）运行结果"></a>4）运行结果</h5><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"> ---- @Cacheable test ----</span><br><span class="line">query username from db, id=st1</span><br><span class="line">query username from db, id=st2</span><br><span class="line">query username from db, id=st3</span><br><span class="line">query username from db, id=st4</span><br><span class="line"></span><br><span class="line"> ---- @CachePut test ----</span><br><span class="line">query username from db, id=st1</span><br><span class="line">query username from db, id=st1</span><br><span class="line">query username from db, id=st1</span><br><span class="line">query username from db, id=st2</span><br><span class="line">query username from db, id=st3</span><br><span class="line">query username from db, id=st4</span><br><span class="line">query username from db, id=st4</span><br><span class="line"> </span><br><span class="line"> ---- @CacheEvict test ----</span><br><span class="line">delete username from db, id=st1</span><br><span class="line">query username from db, id=st1</span><br></pre></td></tr></table></figure><h3 id="五、spring-Cache高级扩展-Redis"><a href="#五、spring-Cache高级扩展-Redis" class="headerlink" title="五、spring Cache高级扩展(Redis)"></a>五、spring Cache高级扩展(Redis)</h3><p>1) 自定义cache实现，实现接口<code>org.springframework.cache.Cache</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br></pre></td><td class="code"><pre><span class="line">package springcache;</span><br><span class="line"></span><br><span class="line">import com.alibaba.fastjson.JSON;</span><br><span class="line">import org.springframework.cache.Cache;</span><br><span class="line">import org.springframework.cache.support.SimpleValueWrapper;</span><br><span class="line">import redis.clients.jedis.Jedis;</span><br><span class="line">import serialization.hessian.HessianUtil;</span><br><span class="line"></span><br><span class="line">import java.util.Set;</span><br><span class="line"></span><br><span class="line">/**</span><br><span class="line"> * <p>spring cache redis实现</p></span><br><span class="line"> *</span><br><span class="line"> * @author: zhengyong Date: 2018/11/23 Time: 上午11:46</span><br><span class="line"> */</span><br><span class="line">public class RedisCustomCache implements Cache {</span><br><span class="line"></span><br><span class="line"> private Jedis jedis;</span><br><span class="line"></span><br><span class="line"> private String name;</span><br><span class="line"></span><br><span class="line"> @Override</span><br><span class="line"> public String getName() {</span><br><span class="line"> return this.name;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> @Override</span><br><span class="line"> public Object getNativeCache() {</span><br><span class="line"> return this.jedis;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> @Override</span><br><span class="line"> public ValueWrapper get(Object key) {</span><br><span class="line"> byte[] keyByte = getKeyByte(key);</span><br><span class="line"> byte[] value = jedis.get(keyByte);</span><br><span class="line"> if (value == null) {</span><br><span class="line"> return null;</span><br><span class="line"> }</span><br><span class="line"> Object object = null;</span><br><span class="line"> try {</span><br><span class="line"> object = HessianUtil.decoder(value);</span><br><span class="line"> } catch (Exception e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> }</span><br><span class="line"> System.out.println("get data from redis, key=" + key + "; value=" + JSON.toJSONString(object));</span><br><span class="line"> ValueWrapper obj = (object != null ? new SimpleValueWrapper(object) : null);</span><br><span class="line"> return obj;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> @Override</span><br><span class="line"> public void put(Object key, Object value) {</span><br><span class="line"> if (value == null) {</span><br><span class="line"> return;</span><br><span class="line"> }</span><br><span class="line"> byte[] keyByte = getKeyByte(key);</span><br><span class="line"> byte[] encoderValue = new byte[0];</span><br><span class="line"> System.out.println("put data into redis, key=" + key + "; value=" + JSON.toJSONString(value));</span><br><span class="line"> try {</span><br><span class="line"> encoderValue = HessianUtil.encoder(value);</span><br><span class="line"> } catch (Exception e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> }</span><br><span class="line"> jedis.set(keyByte, encoderValue);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> @Override</span><br><span class="line"> public void evict(Object key) {</span><br><span class="line"> byte[] keyByte = getKeyByte(key);</span><br><span class="line"> jedis.del(keyByte);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> @Override</span><br><span class="line"> public void clear() {</span><br><span class="line"> // TODO</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> private byte[] getKeyByte(Object key) {</span><br><span class="line"> return key.toString().getBytes();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public void setName(String name) {</span><br><span class="line"> this.name = name;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> public void setJedis(Jedis jedis) {</span><br><span class="line"> this.jedis = jedis;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>1.1）HessianUtil 序列化工具类实现：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br></pre></td><td class="code"><pre><span class="line">package serialization.hessian;</span><br><span class="line"></span><br><span class="line">import java.io.ByteArrayInputStream;</span><br><span class="line">import java.io.ByteArrayOutputStream;</span><br><span class="line"></span><br><span class="line">import com.caucho.hessian.io.Hessian2Input;</span><br><span class="line">import com.caucho.hessian.io.Hessian2Output;</span><br><span class="line">import com.caucho.hessian.io.SerializerFactory;</span><br><span class="line"></span><br><span class="line">/**</span><br><span class="line"> * <p></span><br><span class="line"> * hessian2 序列化和反序列化</span><br><span class="line"> * </p></span><br><span class="line"> * Created by zhengyong on 17/5/27.</span><br><span class="line"> */</span><br><span class="line">public class HessianUtil {</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * 序列化数据</span><br><span class="line"> * </span><br><span class="line"> * @param message</span><br><span class="line"> * @throws Exception</span><br><span class="line"> */</span><br><span class="line"> public static byte[] encoder(Object message) throws Exception {</span><br><span class="line"></span><br><span class="line"> ByteArrayOutputStream byteOutputStream = null;</span><br><span class="line"> // hessian解析二进制</span><br><span class="line"> Hessian2Output hessian2Output = null;</span><br><span class="line"> try {</span><br><span class="line"> byteOutputStream = new ByteArrayOutputStream();</span><br><span class="line"> hessian2Output = new Hessian2Output(byteOutputStream);</span><br><span class="line"> /**</span><br><span class="line"> * 设置serializerFactory能将hessian序列化的效率提高几倍，如果不设置会导致最初的几次序列化效率低，出现阻塞的情况。</span><br><span class="line"> * 主要原因是如果hessian2Output中的serializerFactory为空的话，writeObject的时候创建这个对象的时候会出现阻塞，导致最初几次调用耗时过长</span><br><span class="line"> */</span><br><span class="line"> SerializerFactory factory = new SerializerFactory();</span><br><span class="line"> hessian2Output.setSerializerFactory(factory);</span><br><span class="line"> // 写入序列化信息</span><br><span class="line"> // hessian2Output.startMessage();</span><br><span class="line"> hessian2Output.writeObject(message);</span><br><span class="line"> // hessian2Output.completeMessage();</span><br><span class="line"></span><br><span class="line"> hessian2Output.flush(); // 将序列化信息发送出去</span><br><span class="line"></span><br><span class="line"> byte[] data = byteOutputStream.toByteArray();</span><br><span class="line"></span><br><span class="line"> return data;</span><br><span class="line"> } finally {</span><br><span class="line"> if (byteOutputStream != null) {</span><br><span class="line"> byteOutputStream.close();</span><br><span class="line"> }</span><br><span class="line"> if (hessian2Output != null) {</span><br><span class="line"> hessian2Output.close();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * 反序列化二进制数据</span><br><span class="line"> * </span><br><span class="line"> * @param data</span><br><span class="line"> * @return Object</span><br><span class="line"> * @throws Exception</span><br><span class="line"> */</span><br><span class="line"> public static Object decoder(byte[] data) throws Exception {</span><br><span class="line"></span><br><span class="line"> ByteArrayInputStream is = null;</span><br><span class="line"> // hessian解析二进制</span><br><span class="line"> Hessian2Input hessian2Input = null;</span><br><span class="line"> try {</span><br><span class="line"> is = new ByteArrayInputStream(data);</span><br><span class="line"> hessian2Input = new Hessian2Input(is);</span><br><span class="line"> /**</span><br><span class="line"> * 设置serializerFactory能将hessian序列化的效率提高几倍，如果不设置会导致最初的几次序列化效率低，出现阻塞的情况</span><br><span class="line"> */</span><br><span class="line"> SerializerFactory factory = new SerializerFactory();</span><br><span class="line"> hessian2Input.setSerializerFactory(factory);</span><br><span class="line"> // hessian反序列化对象</span><br><span class="line"> //hessian2Input.startMessage();</span><br><span class="line"> Object ret = hessian2Input.readObject();</span><br><span class="line"> //hessian2Input.completeMessage();</span><br><span class="line"> return ret;</span><br><span class="line"> } finally {</span><br><span class="line"> if (is != null) {</span><br><span class="line"> is.close();</span><br><span class="line"> }</span><br><span class="line"> if (hessian2Input != null) {</span><br><span class="line"> hessian2Input.close();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>2) XML修改为如下</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br></pre></td><td class="code"><pre><span class="line"><?xml version="1.0" encoding="UTF-8"?></span><br><span class="line"><beans xmlns="http://www.springframework.org/schema/beans"</span><br><span class="line"> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"</span><br><span class="line"> xmlns:cache="http://www.springframework.org/schema/cache" xmlns:p="http://www.springframework.org/schema/p"</span><br><span class="line"> xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/cache http://www.springframework.org/schema/cache/spring-cache.xsd"></span><br><span class="line"></span><br><span class="line"> <!-- spring cache config--></span><br><span class="line"> <cache:annotation-driven/></span><br><span class="line"> <bean id="userService" class="springcache.UserService"/></span><br><span class="line"> <bean id="redisCustomCache" class="springcache.RedisCustomCache"/></span><br><span class="line"> <!-- redis地址配置 --></span><br><span class="line"> <bean id="jedisBean" class="redis.clients.jedis.Jedis" ></span><br><span class="line"> <constructor-arg name="host" value="127.0.0.1" /></span><br><span class="line"> <constructor-arg name="port" value="6379" /></span><br><span class="line"> <constructor-arg name="timeout" value="100000" /></span><br><span class="line"> </bean></span><br><span class="line"> <bean id="cacheManager" class="org.springframework.cache.support.SimpleCacheManager"></span><br><span class="line"> <property name="caches"></span><br><span class="line"> <set></span><br><span class="line"> <bean class="org.springframework.cache.concurrent.ConcurrentMapCacheFactoryBean" p:name="default"></bean></span><br><span class="line"> <!-- 自定义redis cache实现 --></span><br><span class="line"> <bean class="springcache.RedisCustomCache"></span><br><span class="line"> <property name="jedis" ref="jedisBean"></property></span><br><span class="line"> <property name="name" value="userCache"></property></span><br><span class="line"> </bean></span><br><span class="line"> </set></span><br><span class="line"> </property></span><br><span class="line"> </bean></span><br><span class="line"></span><br><span class="line"></beans></span><br></pre></td></tr></table></figure><h5 id="3）运行结果"><a href="#3）运行结果" class="headerlink" title="3）运行结果"></a>3）运行结果</h5><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br></pre></td><td class="code"><pre><span class="line"> ---- @Cacheable test ----</span><br><span class="line">query username from db, id=st1</span><br><span class="line">put data into redis, key=st1; value="username-st1"</span><br><span class="line">get data from redis, key=st1; value="username-st1"</span><br><span class="line">get data from redis, key=st1; value="username-st1"</span><br><span class="line">query username from db, id=st2</span><br><span class="line">put data into redis, key=st2; value="username-st2"</span><br><span class="line">put data into redis, key=st3; value="username-st3"</span><br><span class="line">query username from db, id=st4</span><br><span class="line">put data into redis, key=st4; value="username-st4"</span><br><span class="line">query username from db, id=st4</span><br><span class="line">get data from redis, key=st4; value="username-st4"</span><br><span class="line"> </span><br><span class="line"> ---- @CachePut test ----</span><br><span class="line">query username from db, id=st1</span><br><span class="line">put data into redis, key=st1; value="username-st1"</span><br><span class="line">query username from db, id=st1</span><br><span class="line">put data into redis, key=st1; value="username-st1"</span><br><span class="line">query username from db, id=st1</span><br><span class="line">put data into redis, key=st1; value="username-st1"</span><br><span class="line">query username from db, id=st2</span><br><span class="line">put data into redis, key=st2; value="username-st2"</span><br><span class="line">query username from db, id=st3</span><br><span class="line">put data into redis, key=st3; value="username-st3"</span><br><span class="line">query username from db, id=st4</span><br><span class="line">put data into redis, key=st4; value="username-st4"</span><br><span class="line">query username from db, id=st4</span><br><span class="line">put data into redis, key=st4; value="username-st4"</span><br><span class="line"> </span><br><span class="line"> ---- @CacheEvict test ----</span><br><span class="line">get data from redis, key=st1; value="username-st1"</span><br><span class="line">delete username from db, id=st1</span><br><span class="line">query username from db, id=st1</span><br><span class="line">put data into redis, key=st1; value="username-st1"</span><br></pre></td></tr></table></figure><h3 id="六、参考地址"><a href="#六、参考地址" class="headerlink" title="六、参考地址"></a>六、参考地址</h3><p>1）<a href="https://www.ibm.com/developerworks/cn/opensource/os-cn-spring-cache/index.html" target="_blank" rel="noopener">https://www.ibm.com/developerworks/cn/opensource/os-cn-spring-cache/index.html</a> </p><h3 id="七、项目源码"><a href="#七、项目源码" class="headerlink" title="七、项目源码"></a>七、项目源码</h3><p><a href="https://github.com/zyongjava/pomelo/blob/master/src/main/java/springcache/SpringCacheTest.java" target="_blank" rel="noopener">https://github.com/zyongjava/pomelo/blob/master/src/main/java/springcache/SpringCacheTest.java</a></p>]]></content>
<summary type="html">
<h3 id="一、概述"><a href="#一、概述" class="headerlink" title="一、概述"></a>一、概述</h3><figure class="highlight plain"><table><tr><td class="gutter"><pr
</summary>
</entry>
<entry>
<title>spring boot 全局Controller错误处理器</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/11/14/spring-boot-%E5%85%A8%E5%B1%80Controller%E9%94%99%E8%AF%AF%E5%A4%84%E7%90%86%E5%99%A8/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/11/14/spring-boot-全局Controller错误处理器/</id>
<published>2018-11-14T08:57:35.000Z</published>
<updated>2018-11-14T09:27:05.135Z</updated>
<content type="html"><![CDATA[<h4 id="背景"><a href="#背景" class="headerlink" title="背景"></a>背景</h4><p>SpringBoot默认有自定义异常处理的体系，在做SpringBoot项目的时候，如果是抛出了运行时异常，springBoot并会对异常进行处理;</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">1. json接口返回如下异常信息:</span><br><span class="line">{</span><br><span class="line"> "timestamp": 1478571808052,</span><br><span class="line"> "status": 404,</span><br><span class="line"> "error": "Not Found",</span><br><span class="line"> "message": "No message available",</span><br><span class="line"> "path": "/json/rpc"</span><br><span class="line">}</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">2. html接口返回如下信息:</span><br><span class="line"></span><br><span class="line">Whitelabel Error Page</span><br><span class="line"></span><br><span class="line">This application has no explicit mapping for /html/rpc, so you are seeing this as a fallback.</span><br><span class="line"></span><br><span class="line">Thu Sep 07 16:27:43 CST 2018</span><br><span class="line">There was an unexpected error (type=Not Found, status=404).</span><br><span class="line">No message available</span><br></pre></td></tr></table></figure><p>追究其原因，发现SpirngBoot出现异常信息时候，会默认访问/error,springBoot种有BasicErrorController这个类来处理异常信息.</p><h4 id="改造"><a href="#改造" class="headerlink" title="改造"></a>改造</h4><p>我们通过继承<code>ErrorController</code>,实现自己的处理器</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br></pre></td><td class="code"><pre><span class="line">package cn.pomelo.controller;</span><br><span class="line"></span><br><span class="line">import java.util.Map;</span><br><span class="line"></span><br><span class="line">import javax.servlet.http.HttpServletRequest;</span><br><span class="line"></span><br><span class="line">import org.slf4j.Logger;</span><br><span class="line">import org.slf4j.LoggerFactory;</span><br><span class="line">import org.springframework.beans.factory.annotation.Autowired;</span><br><span class="line">import org.springframework.boot.autoconfigure.web.ErrorAttributes;</span><br><span class="line">import org.springframework.boot.autoconfigure.web.ErrorController;</span><br><span class="line">import org.springframework.http.HttpStatus;</span><br><span class="line">import org.springframework.http.MediaType;</span><br><span class="line">import org.springframework.http.ResponseEntity;</span><br><span class="line">import org.springframework.stereotype.Controller;</span><br><span class="line">import org.springframework.web.bind.annotation.RequestMapping;</span><br><span class="line">import org.springframework.web.bind.annotation.ResponseBody;</span><br><span class="line">import org.springframework.web.context.request.RequestAttributes;</span><br><span class="line">import org.springframework.web.context.request.ServletRequestAttributes;</span><br><span class="line">import org.springframework.web.servlet.ModelAndView;</span><br><span class="line">import com.alibaba.fastjson.JSON;</span><br><span class="line"></span><br><span class="line">/**</span><br><span class="line"> * 统一错误处理。<br/></span><br><span class="line"> * </span><br><span class="line"> * @see {http://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#howto-customize-the-whitelabel-error-page}</span><br><span class="line"> * @see {http://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#boot-features-error-handling} <br/></span><br><span class="line"> */</span><br><span class="line">@Controller</span><br><span class="line">public class CommonErrorController implements ErrorController {</span><br><span class="line"></span><br><span class="line"> private static final Logger logger = LoggerFactory.getLogger(CommonErrorController.class);</span><br><span class="line"></span><br><span class="line"> public final static String ERROR_PATH = "/error";</span><br><span class="line"></span><br><span class="line"> @Autowired</span><br><span class="line"> private ErrorAttributes errorAttributes;</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * Supports other formats JSON</span><br><span class="line"> * </span><br><span class="line"> * @param request HttpServletRequest</span><br><span class="line"> * @return ResponseEntity</span><br><span class="line"> */</span><br><span class="line"> @RequestMapping(value = ERROR_PATH, produces = MediaType.APPLICATION_JSON_UTF8_VALUE)</span><br><span class="line"> @ResponseBody</span><br><span class="line"> public String error(HttpServletRequest request) {</span><br><span class="line"> Map<String, Object> body = getErrorAttributes(request);</span><br><span class="line"> HttpStatus status = getStatus(request);</span><br><span class="line"> return JSON.toJSONString(new ResponseEntity<>(body, status).getBody());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * Supports the HTML Error View</span><br><span class="line"> * </span><br><span class="line"> * @param request HttpServletRequest</span><br><span class="line"> * @return ModelAndView</span><br><span class="line"> */</span><br><span class="line"> @RequestMapping(value = ERROR_PATH, produces = MediaType.TEXT_HTML_VALUE)</span><br><span class="line"> public ModelAndView errorHtml(HttpServletRequest request) {</span><br><span class="line"> return new ModelAndView("globalError", getErrorAttributes(request));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * 错误信息详情map</span><br><span class="line"> * </span><br><span class="line"> * @param request</span><br><span class="line"> * @return ErrorAttributes</span><br><span class="line"> */</span><br><span class="line"> private Map<String, Object> getErrorAttributes(HttpServletRequest request) {</span><br><span class="line"> RequestAttributes requestAttributes = new ServletRequestAttributes(request);</span><br><span class="line"> Map<String, Object> map = errorAttributes.getErrorAttributes(requestAttributes, isTrace(request));</span><br><span class="line"> if (request.getAttribute("status") instanceof Integer) {</span><br><span class="line"> map.put("status", request.getAttribute("status"));</span><br><span class="line"> }</span><br><span class="line"> map.put("url", request.getRequestURL().toString());</span><br><span class="line"> map.putIfAbsent("path", request.getAttribute("raw_url"));</span><br><span class="line"> if (logger.isInfoEnabled()) {</span><br><span class="line"> logger.info("error occurred, status={}, path={}, msg={}, error={}, exception={}", map.get("status"),</span><br><span class="line"> map.get("path"), map.get("message"), map.get("error"), map.get("exception"));</span><br><span class="line"> }</span><br><span class="line"> return map;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * 获取状态码</span><br><span class="line"> * </span><br><span class="line"> * @param request HttpServletRequest</span><br><span class="line"> * @return HttpStatus</span><br><span class="line"> */</span><br><span class="line"> private HttpStatus getStatus(HttpServletRequest request) {</span><br><span class="line"> Integer statusCode = (Integer) request.getAttribute("javax.servlet.error.status_code");</span><br><span class="line"> if (statusCode == null) {</span><br><span class="line"> return HttpStatus.INTERNAL_SERVER_ERROR;</span><br><span class="line"> }</span><br><span class="line"> return HttpStatus.valueOf(statusCode);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * 是否打印错误信息</span><br><span class="line"> * </span><br><span class="line"> * @param request HttpServletRequest</span><br><span class="line"> * @return if trace error</span><br><span class="line"> */</span><br><span class="line"> private boolean isTrace(HttpServletRequest request) {</span><br><span class="line"> String parameter = request.getParameter("trace");</span><br><span class="line"> if (parameter == null) {</span><br><span class="line"> return false;</span><br><span class="line"> }</span><br><span class="line"> return !"false".equals(parameter.toLowerCase());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> @Override</span><br><span class="line"> public String getErrorPath() {</span><br><span class="line"> return ERROR_PATH;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h5 id="修改后结果："><a href="#修改后结果：" class="headerlink" title="修改后结果："></a>修改后结果：</h5><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">1. json返回：</span><br><span class="line">{</span><br><span class="line"> "status": 404</span><br><span class="line"> "url": "/json/rpc",</span><br><span class="line"> "path": "/json/rpc?p=1"</span><br><span class="line">}</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">2. html页面返回：</span><br><span class="line">渲染视图：globalError.html</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<h4 id="背景"><a href="#背景" class="headerlink" title="背景"></a>背景</h4><p>SpringBoot默认有自定义异常处理的体系，在做SpringBoot项目的时候，如果是抛出了运行时异常，springBoot并会对异常进
</summary>
</entry>
<entry>
<title>redis分布式锁</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/11/12/redis%E5%88%86%E5%B8%83%E5%BC%8F%E9%94%81/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/11/12/redis分布式锁/</id>
<published>2018-11-12T09:00:17.000Z</published>
<updated>2018-11-12T09:02:01.111Z</updated>
<content type="html"><![CDATA[<h5 id="工具类："><a href="#工具类：" class="headerlink" title="工具类："></a>工具类：</h5><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br></pre></td><td class="code"><pre><span class="line">package redis;</span><br><span class="line"></span><br><span class="line">import org.apache.commons.lang3.math.NumberUtils;</span><br><span class="line">import redis.clients.jedis.Jedis;</span><br><span class="line"></span><br><span class="line">import java.util.Objects;</span><br><span class="line"></span><br><span class="line">/**</span><br><span class="line"> * <pre>https://blog.csdn.net/rich_family/article/details/79034808</pre></span><br><span class="line"> *</span><br><span class="line"> * @author: zhengyong Date: 2018/7/6 Time: 下午5:32</span><br><span class="line"> */</span><br><span class="line">public class RedisDistributedLock {</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * 获取锁直到成功</span><br><span class="line"> *</span><br><span class="line"> * @param jedis redis客户端</span><br><span class="line"> * @param key 锁key</span><br><span class="line"> * @param timeout 锁超时时间</span><br><span class="line"> */</span><br><span class="line"> public void tryLockSuccess(Jedis jedis, String key, long timeout) {</span><br><span class="line"> // 一直循环直到获取到锁</span><br><span class="line"> while (!this.tryLock(jedis, key, timeout)) {</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * 获取锁</span><br><span class="line"> *</span><br><span class="line"> * @param jedis redis客户端</span><br><span class="line"> * @param key 锁key</span><br><span class="line"> * @param timeout 锁超时时间</span><br><span class="line"> * @return true / false</span><br><span class="line"> */</span><br><span class="line"> public boolean tryLock(Jedis jedis, String key, long timeout) {</span><br><span class="line"> long currentTime = System.currentTimeMillis();</span><br><span class="line"> String expires = String.valueOf(timeout + currentTime);</span><br><span class="line"> //设置互斥量</span><br><span class="line"> if (jedis.setnx(key, expires) > 0) {</span><br><span class="line"> return true;</span><br><span class="line"> } else {</span><br><span class="line"> // 未获取到锁，判断锁是否超时</span><br><span class="line"> String currentLockTime = jedis.get(key);</span><br><span class="line"> //检查锁是否超时, 如果超时，尝试获取锁</span><br><span class="line"> if (currentLockTime != null && Long.parseLong(currentLockTime) < currentTime) {</span><br><span class="line"> //获取旧的锁时间并设置互斥量</span><br><span class="line"> String oldLockTime = jedis.getSet(key, expires);</span><br><span class="line"> //旧值与当前时间比较</span><br><span class="line"> if (oldLockTime != null && Objects.equals(oldLockTime, currentLockTime)) {</span><br><span class="line"> return true;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> return false;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> /**</span><br><span class="line"> * 释放锁（如果锁已超时，那么锁可能已由其他线程获得，这时直接执行 DEL lock.id 操作会导致把其他线程已获得的锁释放掉。所以这里只删除未超时的lock.id）</span><br><span class="line"> *</span><br><span class="line"> * @param jedis redis客户端</span><br><span class="line"> * @param key 锁key</span><br><span class="line"> * @return true / false</span><br><span class="line"> */</span><br><span class="line"> public boolean unlock(Jedis jedis, String key) {</span><br><span class="line"> //判断锁是否超时，没有超时才将互斥量删除</span><br><span class="line"> String expire = jedis.get(key);</span><br><span class="line"> long lockExpiresTime = NumberUtils.toLong(expire);</span><br><span class="line"> if (lockExpiresTime > System.currentTimeMillis()) {</span><br><span class="line"> jedis.del(key);</span><br><span class="line"> }</span><br><span class="line"> return true;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure><h5 id="测试类："><a href="#测试类：" class="headerlink" title="测试类："></a>测试类：</h5><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br></pre></td><td class="code"><pre><span class="line">package redis;</span><br><span class="line"></span><br><span class="line">import redis.clients.jedis.Jedis;</span><br><span class="line"></span><br><span class="line">/**</span><br><span class="line"> * @author: zhengyong Date: 2018/7/6 Time: 下午5:56</span><br><span class="line"> */</span><br><span class="line">public class RedisTest {</span><br><span class="line"> /**</span><br><span class="line"> * redis ip</span><br><span class="line"> */</span><br><span class="line"> private static final String redisHost = "127.0.0.1";</span><br><span class="line"> /**</span><br><span class="line"> * redis 端口</span><br><span class="line"> */</span><br><span class="line"> private static final int redisPort = 6379;</span><br><span class="line"> /**</span><br><span class="line"> * redis读取超时时间</span><br><span class="line"> */</span><br><span class="line"> private static final int redisTimeout = 1000000;</span><br><span class="line"> /**</span><br><span class="line"> * 锁超时2秒</span><br><span class="line"> */</span><br><span class="line"> private static final long lockTimeout = 2000;</span><br><span class="line"></span><br><span class="line"> private static RedisDistributedLock lock = new RedisDistributedLock();</span><br><span class="line"></span><br><span class="line"> private static int count = 0;</span><br><span class="line"></span><br><span class="line"> public static void main(String[] args) throws InterruptedException {</span><br><span class="line"> // 多线程测试分布式锁</span><br><span class="line"> for (int i = 0; i < 100; i++) {</span><br><span class="line"> new Thread(new Runnable() {</span><br><span class="line"> @Override</span><br><span class="line"> public void run() {</span><br><span class="line"> Jedis jedis = new Jedis(redisHost, redisPort, redisTimeout);</span><br><span class="line"> execute(jedis);</span><br><span class="line"> }</span><br><span class="line"> }).start();</span><br><span class="line"> }</span><br><span class="line"> Thread.sleep(3000);</span><br><span class="line"> System.out.println(count);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> private static void execute(Jedis jedis) {</span><br><span class="line"> String key = "execute";</span><br><span class="line"> // 获取分布式锁</span><br><span class="line"> lock.tryLockSuccess(jedis, key, lockTimeout);</span><br><span class="line"> try {</span><br><span class="line"> count++;</span><br><span class="line"> } finally {</span><br><span class="line"> lock.unlock(jedis, key);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<h5 id="工具类："><a href="#工具类：" class="headerlink" title="工具类："></a>工具类：</h5><figure class="highlight plain"><table><tr><td class="gutter"><pr
</summary>
</entry>
<entry>
<title>java8-lambda表达式之stream</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/11/12/java8-lambda%E8%A1%A8%E8%BE%BE%E5%BC%8F%E4%B9%8Bstream/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/11/12/java8-lambda表达式之stream/</id>
<published>2018-11-12T07:52:47.000Z</published>
<updated>2018-11-12T07:54:48.775Z</updated>
<content type="html"><![CDATA[<h4 id="Stream-API"><a href="#Stream-API" class="headerlink" title="Stream API"></a>Stream API</h4><p>新添加的Stream API（java.util.stream） 把真正的函数式编程风格引入到Java中。</p><h5 id="示列代码："><a href="#示列代码：" class="headerlink" title="示列代码："></a>示列代码：</h5><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br></pre></td><td class="code"><pre><span class="line">package java8.lambda;</span><br><span class="line"></span><br><span class="line">import java.util.Arrays;</span><br><span class="line">import java.util.IntSummaryStatistics;</span><br><span class="line">import java.util.List;</span><br><span class="line">import java.util.stream.Collectors;</span><br><span class="line"></span><br><span class="line">/**</span><br><span class="line"> * java 8 lambada 基础操作（http://www.runoob.com/java/java8-streams.html）</span><br><span class="line"> *</span><br><span class="line"> * @author: zhengyong Date: 2018/11/1 Time: 上午10:49</span><br><span class="line"> */</span><br><span class="line">public class Test {</span><br><span class="line"></span><br><span class="line"> public static void main(String[] args) {</span><br><span class="line"></span><br><span class="line"> //Runnable是一个函数接口，只包含了有个无参数的，返回void的run方法；</span><br><span class="line"> //所以lambda表达式左边没有参数，右边也没有return，只是单纯的打印一句话</span><br><span class="line"> new Thread(() -> System.out.println("Test Java8 lambda!")).start();</span><br><span class="line"></span><br><span class="line"> //In Java 8:</span><br><span class="line"> List<String> features = Arrays.asList("Lambdas", "Default Method", "Stream API", "Date and Time API");</span><br><span class="line"></span><br><span class="line"> // 1. forEach: 迭代流中的每个数据</span><br><span class="line"> System.out.println("\n... 1. start forEach");</span><br><span class="line"> features.forEach(n -> System.out.println(n));</span><br><span class="line"></span><br><span class="line"> System.out.println("\n... 2. start stream ");</span><br><span class="line"> System.out.println("... 3. start map");</span><br><span class="line"> // 2. stream: 为集合创建串行流</span><br><span class="line"> // 3. map: 映射每个元素到对应的结果</span><br><span class="line"> features.stream().map((s) -> s + "***").forEach(System.out::println);</span><br><span class="line"></span><br><span class="line"> // 4. filter: 用于通过设置的条件过滤出元素</span><br><span class="line"> System.out.println("\n... 4. start filter");</span><br><span class="line"> features.stream().filter(f -> f.startsWith("L")).forEach(System.out::println);</span><br><span class="line"></span><br><span class="line"> // 5. limit: 用于获取指定数量的流</span><br><span class="line"> System.out.println("\n... 5. start limit");</span><br><span class="line"> features.stream().limit(2).forEach(System.out::println);</span><br><span class="line"></span><br><span class="line"> // 6. sorted: 用于对流进行排序</span><br><span class="line"> System.out.println("\n... 6. start sorted");</span><br><span class="line"> features.stream().sorted().forEach(System.out::println);</span><br><span class="line"></span><br><span class="line"> // 7. parallel: 为集合创建并行流</span><br><span class="line"> System.out.println("\n... 7. start parallel stream");</span><br><span class="line"> features.parallelStream().filter(f -> f.startsWith("L")).forEach(System.out::println);</span><br><span class="line"></span><br><span class="line"> // 8.Collectors :实现了很多归约操作</span><br><span class="line"> System.out.println("\n... 8. start Collectors");</span><br><span class="line"> List<String> list = features.stream().filter(f -> f.startsWith("L")).collect(Collectors.toList());</span><br><span class="line"> String str = features.stream().filter(f -> !f.isEmpty()).collect(Collectors.joining(","));</span><br><span class="line"> System.out.println("Collectors list=" + list);</span><br><span class="line"> System.out.println("Collectors str=" + str);</span><br><span class="line"></span><br><span class="line"> // 9. summaryStatistics: 统计结果的收集器也非常有用。它们主要用于int、double、long等基本类型上</span><br><span class="line"> System.out.println("\n... 8. start summaryStatistics");</span><br><span class="line"> List<Integer> numbers = Arrays.asList(3, 2, 2, 3, 7, 3, 5);</span><br><span class="line"> IntSummaryStatistics stats = numbers.stream().mapToInt(x -> x).summaryStatistics();</span><br><span class="line"> System.out.println("列表中最大的数 : " + stats.getMax());</span><br><span class="line"> System.out.println("列表中最小的数 : " + stats.getMin());</span><br><span class="line"> System.out.println("所有数之和 : " + stats.getSum());</span><br><span class="line"> System.out.println("平均数 : " + stats.getAverage());</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h5 id="运行结果："><a href="#运行结果：" class="headerlink" title="运行结果："></a>运行结果：</h5><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br></pre></td><td class="code"><pre><span class="line">Test Java8 lambda!</span><br><span class="line"></span><br><span class="line">... 1. start forEach</span><br><span class="line">Lambdas</span><br><span class="line">Default Method</span><br><span class="line">Stream API</span><br><span class="line">Date and Time API</span><br><span class="line"></span><br><span class="line">... 2. start stream </span><br><span class="line">... 3. start map</span><br><span class="line">Lambdas***</span><br><span class="line">Default Method***</span><br><span class="line">Stream API***</span><br><span class="line">Date and Time API***</span><br><span class="line"></span><br><span class="line">... 4. start filter</span><br><span class="line">Lambdas</span><br><span class="line"></span><br><span class="line">... 5. start limit</span><br><span class="line">Lambdas</span><br><span class="line">Default Method</span><br><span class="line"></span><br><span class="line">... 6. start sorted</span><br><span class="line">Date and Time API</span><br><span class="line">Default Method</span><br><span class="line">Lambdas</span><br><span class="line">Stream API</span><br><span class="line"></span><br><span class="line">... 7. start parallel stream</span><br><span class="line">Lambdas</span><br><span class="line"></span><br><span class="line">... 8. start Collectors</span><br><span class="line">Collectors list=[Lambdas]</span><br><span class="line">Collectors str=Lambdas,Default Method,Stream API,Date and Time API</span><br><span class="line"></span><br><span class="line">... 8. start summaryStatistics</span><br><span class="line">列表中最大的数 : 7</span><br><span class="line">列表中最小的数 : 2</span><br><span class="line">所有数之和 : 25</span><br><span class="line">平均数 : 3.5714285714285716</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<h4 id="Stream-API"><a href="#Stream-API" class="headerlink" title="Stream API"></a>Stream API</h4><p>新添加的Stream API（java.util.stream） 把真正的函
</summary>
</entry>
<entry>
<title>awk使用</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/09/05/awk%E4%BD%BF%E7%94%A8/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/09/05/awk使用/</id>
<published>2018-09-05T07:49:34.000Z</published>
<updated>2018-09-05T08:52:24.562Z</updated>
<content type="html"><![CDATA[<p>首先建立文本<code>log.txt</code>:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">2 this is a test</span><br><span class="line">3 Are you like awk</span><br><span class="line">This's a test</span><br><span class="line">10 There are orange,apple,mongo</span><br></pre></td></tr></table></figure><h4 id="A-每行按空格或TAB分割，输出文本中的1、4项-例如：awk-39-print-1-4-39-log-txt"><a href="#A-每行按空格或TAB分割，输出文本中的1、4项-例如：awk-39-print-1-4-39-log-txt" class="headerlink" title="A. 每行按空格或TAB分割，输出文本中的1、4项, 例如：awk '{print $1,$4}' log.txt"></a>A. 每行按空格或TAB分割，输出文本中的1、4项, 例如：<code>awk '{print $1,$4}' log.txt</code></h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">$ awk '{print $1,$4}' log.txt</span><br><span class="line"></span><br><span class="line">2 a</span><br><span class="line">3 like</span><br><span class="line">This's</span><br><span class="line">10 orange,apple,mongo</span><br></pre></td></tr></table></figure><h4 id="B-使用”-”分割，例如：awk-F-39-print-1-2-39-log-txt"><a href="#B-使用”-”分割，例如：awk-F-39-print-1-2-39-log-txt" class="headerlink" title="B. 使用”,”分割，例如：awk -F, '{print $1,$2}' log.txt"></a>B. 使用”,”分割，例如：<code>awk -F, '{print $1,$2}' log.txt</code></h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">$ awk -F, '{print $1,$2}' log.txt</span><br><span class="line"></span><br><span class="line">2 this is a test</span><br><span class="line">3 Are you like awk</span><br><span class="line">This's a test</span><br><span class="line">10 There are orange apple</span><br></pre></td></tr></table></figure><h4 id="C-使用多个分隔符-先使用空格分割，然后对分割结果再使用”-”分割-例如：awk-F-39-39-39-print-1-2-5-39-log-txt"><a href="#C-使用多个分隔符-先使用空格分割，然后对分割结果再使用”-”分割-例如：awk-F-39-39-39-print-1-2-5-39-log-txt" class="headerlink" title="C. 使用多个分隔符.先使用空格分割，然后对分割结果再使用”,”分割. 例如：awk -F '[ ,]' '{print $1,$2,$5}' log.txt"></a>C. 使用多个分隔符.先使用空格分割，然后对分割结果再使用”,”分割. 例如：<code>awk -F '[ ,]' '{print $1,$2,$5}' log.txt</code></h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">$ awk -F '[ ,]' '{print $1,$2,$5}' log.txt</span><br><span class="line"></span><br><span class="line">2 this test</span><br><span class="line">3 Are awk</span><br><span class="line">This's a</span><br><span class="line">10 There apple</span><br></pre></td></tr></table></figure><h4 id="D-使用awk-v设置变量a-1-例如：awk-v-a-1-39-print-1-1-a-39-log-txt"><a href="#D-使用awk-v设置变量a-1-例如：awk-v-a-1-39-print-1-1-a-39-log-txt" class="headerlink" title="D. 使用awk -v设置变量a=1, 例如：awk -v a=1 '{print $1,$1+a}' log.txt"></a>D. 使用<code>awk -v</code>设置变量<code>a=1</code>, 例如：<code>awk -v a=1 '{print $1,$1+a}' log.txt</code></h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">$ awk -v a=1 '{print $1,$1+a}' log.txt</span><br><span class="line"></span><br><span class="line">2 3</span><br><span class="line">3 4</span><br><span class="line">This's 1</span><br><span class="line">10 11</span><br></pre></td></tr></table></figure><h4 id="E-使用脚本，例如：awk-f-cal-awk-log-txt"><a href="#E-使用脚本，例如：awk-f-cal-awk-log-txt" class="headerlink" title="E. 使用脚本，例如：awk -f cal.awk log.txt"></a>E. 使用脚本，例如：<code>awk -f cal.awk log.txt</code></h4><h4 id="F-awk使用运算符"><a href="#F-awk使用运算符" class="headerlink" title="F. awk使用运算符"></a>F. awk使用运算符</h4><p>1) 过滤第一例大于2的行. 例如：<code>awk '$1>2' log.txt</code></p><p>2) 过滤第一列大于2并且第二列等于’Are’的行. 例如：<code>awk '$1>2 && $2=="Are" {print $1,$2,$3}' log.txt</code></p><h4 id="G-awk正使用则表达式"><a href="#G-awk正使用则表达式" class="headerlink" title="G. awk正使用则表达式"></a>G. awk正使用则表达式</h4><p>注意：~ 表示模式开始。// 中是模式。</p><p>例子：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"># 输出第二列包含 "th"，并打印第二列与第四列</span><br><span class="line">$ awk '$2 ~ /th/ {print $2,$4}' log.txt</span><br><span class="line">---------------------------------------------</span><br><span class="line">this a</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"># 输出包含"re" 的行</span><br><span class="line">$ awk '/re/ ' log.txt</span><br><span class="line">---------------------------------------------</span><br><span class="line">3 Are you like awk</span><br><span class="line">10 There are orange,apple,mongo</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"># 忽略大小写</span><br><span class="line">$ awk 'BEGIN{IGNORECASE=1} /this/' log.txt</span><br><span class="line">---------------------------------------------</span><br><span class="line">2 this is a test</span><br><span class="line">This's a test</span><br></pre></td></tr></table></figure><h4 id="H-awk脚本"><a href="#H-awk脚本" class="headerlink" title="H. awk脚本"></a>H. awk脚本</h4><p>BEGIN{ 这里面放的是执行前的语句 }</p><p>END {这里面放的是处理完所有的行后要执行的语句 }</p><p>{这里面放的是处理每一行时要执行的语句}</p><p>例如<code>cal.awk</code>内容如下：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line">#!/bin/awk -f</span><br><span class="line">#运行前</span><br><span class="line">BEGIN {</span><br><span class="line"> math = 0</span><br><span class="line"> english = 0</span><br><span class="line"> computer = 0</span><br><span class="line"> </span><br><span class="line"> printf "NAME NO. MATH ENGLISH COMPUTER TOTAL\n"</span><br><span class="line"> printf "---------------------------------------------\n"</span><br><span class="line">}</span><br><span class="line">#运行中</span><br><span class="line">{</span><br><span class="line"> math+=$3</span><br><span class="line"> english+=$4</span><br><span class="line"> computer+=$5</span><br><span class="line"> printf "%-6s %-6s %4d %8d %8d %8d\n", $1, $2, $3,$4,$5, $3+$4+$5</span><br><span class="line">}</span><br><span class="line">#运行后</span><br><span class="line">END {</span><br><span class="line"> printf "---------------------------------------------\n"</span><br><span class="line"> printf " TOTAL:%10d %8d %8d \n", math, english, computer</span><br><span class="line"> printf "AVERAGE:%10.2f %8.2f %8.2f\n", math/NR, english/NR, computer/NR</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>G. 统计行数</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">awk '/error/' app.log | awk 'END {print NR}'</span><br></pre></td></tr></table></figure><p>或者：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">grep -c 'error' app.log</span><br></pre></td></tr></table></figure><p>参考：<a href="http://www.runoob.com/linux/linux-comm-awk.html" target="_blank" rel="noopener">http://www.runoob.com/linux/linux-comm-awk.html</a></p>]]></content>
<summary type="html">
<p>首先建立文本<code>log.txt</code>:</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span c
</summary>
<category term="awk" scheme="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/tags/awk/"/>
</entry>
<entry>
<title>docker使用</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/09/05/docker%E4%BD%BF%E7%94%A8/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/09/05/docker使用/</id>
<published>2018-09-05T07:46:20.000Z</published>
<updated>2018-09-05T07:53:05.050Z</updated>
<content type="html"><![CDATA[<h4 id="一、简介"><a href="#一、简介" class="headerlink" title="一、简介"></a>一、简介</h4><blockquote><p> Docker 和传统虚拟化方式的不同之处。传统虚拟机技术是虚拟出一套硬件后，在其上运行一个完整操作系统，在该系统上再运行所需应用进程；而容器内的应用进程直接运行于宿主的内核，容器内没有自己的内核，而且也没有进行硬件虚拟。因此容器要比传统虚拟机更为轻便</p></blockquote><blockquote><p>镜像（Image）和容器（Container）的关系，就像是面向对象程序设计中的类和实例一样，镜像是静态的定义，容器是镜像运行时的实体。容器可以被创建、启动、停止、删除、暂停等</p></blockquote><h4 id="二、安装须知"><a href="#二、安装须知" class="headerlink" title="二、安装须知"></a>二、安装须知</h4><p>操作系统: Mac OS Yosemite 10.10.5</p><p>Mac地址：<a href="https://download.docker.com/mac/stable/Docker.dmg" target="_blank" rel="noopener">https://download.docker.com/mac/stable/Docker.dmg</a></p><p>Docker.dmg：安装提供Docker Engine，Docker CLI客户端，Docker Compose和Docker Machine</p><h4 id="三、安装"><a href="#三、安装" class="headerlink" title="三、安装"></a>三、安装</h4><p>1）运行Docker.dmg安装</p><p>2）配置个人加速镜像（我使用阿里云<a href="https://cr.console.aliyun.com/#/accelerator）" target="_blank" rel="noopener">https://cr.console.aliyun.com/#/accelerator）</a></p><p>3) 运行<code>docker version</code>和<code>docker info</code>检查</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">➜ docker version</span><br><span class="line">Client:</span><br><span class="line"> Version: 17.03.0-ce</span><br><span class="line"> API version: 1.26</span><br><span class="line"> Go version: go1.7.5</span><br><span class="line"> Git commit: 60ccb22</span><br><span class="line"> Built: Thu Feb 23 10:40:59 2017</span><br><span class="line"> OS/Arch: darwin/amd64</span><br><span class="line"></span><br><span class="line">Server:</span><br><span class="line"> Version: 17.03.0-ce</span><br><span class="line"> API version: 1.26 (minimum version 1.12)</span><br><span class="line"> Go version: go1.7.5</span><br><span class="line"> Git commit: 3a232c8</span><br><span class="line"> Built: Tue Feb 28 07:52:04 2017</span><br><span class="line"> OS/Arch: linux/amd64</span><br><span class="line"> Experimental: true</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br></pre></td><td class="code"><pre><span class="line">➜ docker info</span><br><span class="line">Containers: 1</span><br><span class="line"> Running: 1</span><br><span class="line"> Paused: 0</span><br><span class="line"> Stopped: 0</span><br><span class="line">Images: 1</span><br><span class="line">Server Version: 17.03.0-ce</span><br><span class="line">Storage Driver: overlay2</span><br><span class="line"> Backing Filesystem: extfs</span><br><span class="line"> Supports d_type: true</span><br><span class="line"> Native Overlay Diff: true</span><br><span class="line">Logging Driver: json-file</span><br><span class="line">Cgroup Driver: cgroupfs</span><br><span class="line">Plugins:</span><br><span class="line"> Volume: local</span><br><span class="line"> Network: bridge host ipvlan macvlan null overlay</span><br><span class="line">Swarm: inactive</span><br><span class="line">Runtimes: runc</span><br><span class="line">Default Runtime: runc</span><br><span class="line">Init Binary: docker-init</span><br><span class="line">containerd version: 977c511eda0925a723debdc94d09459af49d082a</span><br><span class="line">runc version: a01dafd48bc1c7cc12bdb01206f9fea7dd6feb70</span><br><span class="line">init version: 949e6fa</span><br><span class="line">Security Options:</span><br><span class="line"> seccomp</span><br><span class="line"> Profile: default</span><br><span class="line">Kernel Version: 4.9.12-moby</span><br><span class="line">Operating System: Alpine Linux v3.5</span><br><span class="line">OSType: linux</span><br><span class="line">Architecture: x86_64</span><br><span class="line">CPUs: 2</span><br><span class="line">Total Memory: 1.952 GiB</span><br><span class="line">Name: moby</span><br><span class="line">ID: V5PN:CAOJ:YYKT:B7IK:PTP3:OQKS:Q26Q:UJYZ:OG2K:HSPK:WRME:KJP5</span><br><span class="line">Docker Root Dir: /var/lib/docker</span><br><span class="line">Debug Mode (client): false</span><br><span class="line">Debug Mode (server): true</span><br><span class="line"> File Descriptors: 23</span><br><span class="line"> Goroutines: 32</span><br><span class="line"> System Time: 2017-03-27T03:33:06.945792584Z</span><br><span class="line"> EventsListeners: 1</span><br><span class="line">No Proxy: 127.0.0.1</span><br><span class="line">Registry: https://index.docker.io/v1/</span><br><span class="line">Experimental: true</span><br><span class="line">Insecure Registries:</span><br><span class="line"> 127.0.0.0/8</span><br><span class="line">Registry Mirrors:</span><br><span class="line"> https://xxxxx.mirror.aliyuncs.com</span><br><span class="line">Live Restore Enabled: false</span><br></pre></td></tr></table></figure><h4 id="四、运行nginx"><a href="#四、运行nginx" class="headerlink" title="四、运行nginx"></a>四、运行nginx</h4><p>启动运行nginx, 执行<code>docker run -d -p 80:80 --name webserver nginx</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">➜ docker run -d -p 80:80 --name webserver nginx</span><br><span class="line">Unable to find image 'nginx:latest' locally</span><br><span class="line">latest: Pulling from library/nginx</span><br><span class="line">693502eb7dfb: Pull complete</span><br><span class="line">6decb850d2bc: Pull complete</span><br><span class="line">c3e19f087ed6: Pull complete</span><br><span class="line">Digest: sha256:52a189e49c0c797cfc5cbfe578c68c225d160fb13a42954144b29af3fe4fe335</span><br><span class="line">Status: Downloaded newer image for nginx:latest</span><br><span class="line">4b11e6ed897d6df195936f60916ffda704d55f75645270f338a10b68480112e7</span><br></pre></td></tr></table></figure><p>启动bash命令交互操作客户端</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">➜ docker exec -it webserver bash</span><br><span class="line">root@2964bd62006a:/#</span><br></pre></td></tr></table></figure><p>停止niginx, 运行<code>docker stop webserver</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">➜ docker stop webserver</span><br><span class="line">webserver</span><br></pre></td></tr></table></figure><p>删除nginx, 运行<code>docker rm webserver</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">➜ docker rm webserver</span><br><span class="line">webserver</span><br></pre></td></tr></table></figure><h4 id="五、运行ubuntu"><a href="#五、运行ubuntu" class="headerlink" title="五、运行ubuntu"></a>五、运行ubuntu</h4><p>镜像官方仓库：<a href="https://hub.docker.com/explore/" target="_blank" rel="noopener">https://hub.docker.com/explore/</a></p><p>下载ubuntu</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">➜ docker pull ubuntu:14.04</span><br><span class="line">14.04: Pulling from library/ubuntu</span><br><span class="line">30d541b48fc0: Pull complete</span><br><span class="line">8ecd7f80d390: Pull complete</span><br><span class="line">46ec9927bb81: Pull complete</span><br><span class="line">2e67a4d67b44: Pull complete</span><br><span class="line">7d9dd9155488: Pull complete</span><br><span class="line">Digest: sha256:62a5dce5ceccd7f1cb2672a571ebee52cad1f08eec9b57fe4965fb0968a9602e</span><br><span class="line">Status: Downloaded newer image for ubuntu:14.04</span><br></pre></td></tr></table></figure><p>启动运行ubuntu</p><blockquote><p>-it：这是两个参数，一个是 -i：交互式操作，一个是 -t 终端。我们这里打算进入 bash 执行一些命令并查看返回结果，因此我们需要交互式终端。</p><p>–rm：这个参数是说容器退出后随之将其删除。默认情况下，为了排障需求，退出的容器并不会立即删除，除非手动 docker rm。我们这里只是随便执行个命令，看看结果，不需要排障和保留结果，因此使用 –rm 可以避免浪费空间。</p><p>ubuntu:14.04：这是指用 ubuntu:14.04 镜像为基础来启动容器。</p><p>–name myUbuntu：给容器取一个别名叫<code>myUbuntu</code></p></blockquote><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">➜ docker run -it --rm --name myUbuntu ubuntu:14.04</span><br><span class="line">root@2ef73a93ef7b:/#</span><br></pre></td></tr></table></figure><p>或者不指定ubuntu版本，直接启动时，直接下载最新版本<code>ubuntu:latest</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">➜ docker run -it --name myUbuntu ubuntu:14.04</span><br><span class="line">Unable to find image 'ubuntu:latest' locally</span><br><span class="line">latest: Pulling from library/ubuntu</span><br><span class="line">d54efb8db41d: Pull complete</span><br><span class="line">f8b845f45a87: Pull complete</span><br><span class="line">e8db7bf7c39f: Pull complete</span><br><span class="line">9654c40e9079: Pull complete</span><br><span class="line">6d9ef359eaaa: Pull complete</span><br><span class="line">Digest: sha256:dd7808d8792c9841d0b460122f1acf0a2dd1f56404f8d1e56298048885e45535</span><br><span class="line">Status: Downloaded newer image for ubuntu:latest</span><br><span class="line">root@366ea11633cf:/#</span><br></pre></td></tr></table></figure><p>停止ubuntu容器</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">➜ docker stop myUbuntu</span><br><span class="line">myUbuntu</span><br></pre></td></tr></table></figure><p>删除ubuntu容器</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">➜ docker rm myUbuntu</span><br><span class="line">myUbuntu</span><br></pre></td></tr></table></figure><p>查看历史镜像</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">➜ docker images</span><br><span class="line">REPOSITORY TAG IMAGE ID CREATED SIZE</span><br><span class="line">nginx latest 6b914bbcb89e 3 weeks ago 182 MB</span><br><span class="line">ubuntu 14.04 7c09e61e9035 3 weeks ago 188 MB</span><br></pre></td></tr></table></figure><p>删除ubuntu镜像</p><p>使用<code>docker rmi <IMAGE ID></code>删除，但是必须先要<code>docker rm myUbuntu</code>才能删除镜像</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">➜ docker rmi 7c09e61e9035</span><br><span class="line">Untagged: ubuntu:14.04</span><br><span class="line">Untagged: ubuntu@sha256:62a5dce5ceccd7f1cb2672a571ebee52cad1f08eec9b57fe4965fb0968a9602e</span><br><span class="line">Deleted: sha256:7c09e61e90350e8f5c0cba2979003bdfe32c2d027b68b4f0cf9063cdd7b4bafd</span><br><span class="line">Deleted: sha256:304aecb5e13929f85d3ce2e9d83d0212866c8e55a460c94cf24bd75da1c7c153</span><br><span class="line">Deleted: sha256:f302be18d46a45c0edbbd9b4bc02db764a4b0b8cd9bd0490f33dfaff039a3b62</span><br><span class="line">Deleted: sha256:c523f3173f6028e5329fd401331c375f7b9b9e831d915fafaf358f55e36e3747</span><br><span class="line">Deleted: sha256:94e631422130dc414878fd05efe3d59de44c9d8904696a7c299a83f378a92845</span><br><span class="line">Deleted: sha256:c29b5eadf94a90a2abda13e765d4fad4825fd15621dea1d9a98b60b89b835c2a</span><br></pre></td></tr></table></figure><p>查看docker进程</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">➜ docker ps -a</span><br><span class="line">CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES</span><br><span class="line">366ea11633cf ubuntu "/bin/bash" About a minute ago Exited (0) 59 seconds ago myUbuntu</span><br></pre></td></tr></table></figure><h4 id="六、制作自定义nginx"><a href="#六、制作自定义nginx" class="headerlink" title="六、制作自定义nginx"></a>六、制作自定义nginx</h4><p>1) 启动nginx</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">➜ docker run -d -p 80:80 --name webserver nginx</span><br><span class="line">Unable to find image 'nginx:latest' locally</span><br><span class="line">latest: Pulling from library/nginx</span><br><span class="line">693502eb7dfb: Pull complete</span><br><span class="line">6decb850d2bc: Pull complete</span><br><span class="line">c3e19f087ed6: Pull complete</span><br><span class="line">Digest: sha256:52a189e49c0c797cfc5cbfe578c68c225d160fb13a42954144b29af3fe4fe335</span><br><span class="line">Status: Downloaded newer image for nginx:latest</span><br><span class="line">2964bd62006acd43611ea045e8152f651905e213c392e008a59eebe667080b3d</span><br></pre></td></tr></table></figure><p>2） 启动bash控制台并修改<code>index.html</code>内容</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">➜ docker exec -it webserver bash</span><br><span class="line">root@2964bd62006a:/# ls</span><br><span class="line">bin boot devetc home liblib64 media mnt optproc root run sbin srv sys tmp usr var</span><br><span class="line">root@2964bd62006a:/# echo '<h1>Hello, Docker!</h1>' > /usr/share/nginx/html/index.html</span><br><span class="line">root@2964bd62006a:/# exit</span><br><span class="line">exit</span><br></pre></td></tr></table></figure><p>3）查看容器修改内容</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">➜ docker diff webserver</span><br><span class="line">C /root</span><br><span class="line">A /root/.bash_history</span><br><span class="line">C /run</span><br><span class="line">A /run/nginx.pid</span><br><span class="line">C /usr/share/nginx/html/index.html</span><br><span class="line">C /var/cache/nginx</span><br><span class="line">A /var/cache/nginx/client_temp</span><br><span class="line">A /var/cache/nginx/fastcgi_temp</span><br><span class="line">A /var/cache/nginx/proxy_temp</span><br><span class="line">A /var/cache/nginx/scgi_temp</span><br><span class="line">A /var/cache/nginx/uwsgi_temp</span><br></pre></td></tr></table></figure><p>4）制作v2版本tag的nginx</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">➜ docker commit --author "zhengyong" --message "update index.htm" webserver nigix:v2</span><br><span class="line">sha256:b86a71250cf4e41a34af24b7e2924e3909ecdf74262195d6ac30fd57bbdbaf35</span><br></pre></td></tr></table></figure><p>5) 启动v2版本nginx</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">➜ docker run -d -p 80:80 --name ng nigix:v2</span><br><span class="line">72f15356de38c26456cdee8a68651a2de25e59d8f8e4ce31e82c434927ff4b4f</span><br></pre></td></tr></table></figure><h4 id="七、Dockerfile"><a href="#七、Dockerfile" class="headerlink" title="七、Dockerfile"></a>七、Dockerfile</h4><p>1）创建<code>工程目录</code>和<code>Dockerfile</code>文件</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">➜ mkdir mynginx</span><br><span class="line">➜ cd mynginx</span><br><span class="line">➜ touch Dockerfile</span><br></pre></td></tr></table></figure><p>2）<code>Dockerfile</code>文件内容</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">FROM nginx</span><br><span class="line">RUN echo '<h1>Hello, Docker!</h1>' > /usr/share/nginx/html/index.html</span><br></pre></td></tr></table></figure><p>3) 构建docker镜像</p><p>使用命令<code>docker build -t nginx:v3 .</code>构建：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">➜ docker build -t nginx:v3 .</span><br><span class="line">Sending build context to Docker daemon 2.048 kB</span><br><span class="line">Step 1/2 : FROM nginx</span><br><span class="line"> ---> 6b914bbcb89e</span><br><span class="line">Step 2/2 : RUN echo '<h1>Hello, Docker!</h1>' > /usr/share/nginx/html/index.html</span><br><span class="line"> ---> Running in a41235243cb9</span><br><span class="line"> ---> 7a2e14fdfefb</span><br><span class="line">Removing intermediate container a41235243cb9</span><br><span class="line">Successfully built 7a2e14fdfefb</span><br></pre></td></tr></table></figure><p>docker导致mac硬盘空间不足：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">docker rm $(docker ps -a -q)</span><br><span class="line">docker rmi $(docker images -q)</span><br><span class="line">docker volume rm $(docker volume ls |awk '{print $2}')</span><br><span class="line">rm -rf ~/Library/Containers/com.docker.docker/Data/*</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<h4 id="一、简介"><a href="#一、简介" class="headerlink" title="一、简介"></a>一、简介</h4><blockquote>
<p> Docker 和传统虚拟化方式的不同之处。传统虚拟机技术是虚拟出一套硬件后，在其上运行一个完整操
</summary>
<category term="docker" scheme="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/tags/docker/"/>
</entry>
<entry>
<title>Drools使用</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/09/05/Drools%E4%BD%BF%E7%94%A8/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/09/05/Drools使用/</id>
<published>2018-09-05T07:44:42.000Z</published>
<updated>2018-09-05T07:52:55.769Z</updated>
<content type="html"><![CDATA[<h3 id="Maven-dependency"><a href="#Maven-dependency" class="headerlink" title="Maven dependency"></a>Maven dependency</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><dependency></span><br><span class="line"> <groupId>org.kie</groupId></span><br><span class="line"> <artifactId>kie-api</artifactId></span><br><span class="line"> <version>6.5.0.Final</version></span><br><span class="line"></dependency></span><br><span class="line"><dependency></span><br><span class="line"> <groupId>org.drools</groupId></span><br><span class="line"> <artifactId>drools-core</artifactId></span><br><span class="line"> <version>6.5.0.Final</version></span><br><span class="line"></dependency></span><br><span class="line"><dependency></span><br><span class="line"> <groupId>org.drools</groupId></span><br><span class="line"> <artifactId>drools-compiler</artifactId></span><br><span class="line"> <version>6.5.0.Final</version></span><br><span class="line"></dependency></span><br></pre></td></tr></table></figure><h3 id="Rule-Syntax"><a href="#Rule-Syntax" class="headerlink" title="Rule Syntax"></a>Rule Syntax</h3><h5 id="1-Conditions-in-Rules"><a href="#1-Conditions-in-Rules" class="headerlink" title="1. Conditions in Rules"></a>1. Conditions in Rules</h5><p>A rule can contain many conditions and patterns such as:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">Account (balance == 200)</span><br><span class="line">Customer (name == “Vivek”)</span><br></pre></td></tr></table></figure><p>The above conditions check if the Account balance is 200 or the Customer name is “Vivek”.</p><h5 id="2-Variables-in-Rules"><a href="#2-Variables-in-Rules" class="headerlink" title="2. Variables in Rules"></a>2. Variables in Rules</h5><p>A variable name in Drools starts with a Dollar($) symbol.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$account : Account( )</span><br><span class="line">$account is the variable for Account() class</span><br></pre></td></tr></table></figure><p>Drools can work with all the native Java types and even Enum.</p><h5 id="3-Comments-in-Rules"><a href="#3-Comments-in-Rules" class="headerlink" title="3. Comments in Rules"></a>3. Comments in Rules</h5><p>The special characters, # or //, can be used to mark single-line comments.</p><p>For multi-line comments, use the following format:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">/*</span><br><span class="line"> Another line</span><br><span class="line"> .........</span><br><span class="line"> .........</span><br><span class="line">*/</span><br></pre></td></tr></table></figure><h5 id="4-Functions-in-Rules"><a href="#4-Functions-in-Rules" class="headerlink" title="4.Functions in Rules"></a>4.Functions in Rules</h5><p>Functions are a convenience feature. They can be used in conditions and consequences. Functions represent an alternative to the utility/helper classes. For example:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">function double calculateSquare (double value) {</span><br><span class="line"> return value * value;</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h5 id="5-Salience-same-as-priority"><a href="#5-Salience-same-as-priority" class="headerlink" title="5. Salience (same as priority)"></a>5. Salience (same as priority)</h5><p>Salience is a very important feature of Rule Syntax. Salience is used by the conflict resolution strategy to decide which rule to fire first. By default, it is the main criterion.</p><p>We can use salience to define the order of firing rules. Salience has one attribute, which takes any expression that returns a number of type int (positive as well as negative numbers are valid). The higher the value, the more likely a rule will be picked up by the conflict resolution strategy to fire.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">salience ($account.balance * 5)</span><br></pre></td></tr></table></figure><p>The default salience value is 0. We should keep this in mind when assigning salience values to some rules only.</p><h3 id="参考地址"><a href="#参考地址" class="headerlink" title="参考地址"></a>参考地址</h3><p><a href="https://www.tutorialspoint.com/drools/drools_rule_syntax.htm" target="_blank" rel="noopener">https://www.tutorialspoint.com/drools/drools_rule_syntax.htm</a></p>]]></content>
<summary type="html">
<h3 id="Maven-dependency"><a href="#Maven-dependency" class="headerlink" title="Maven dependency"></a>Maven dependency</h3><figure class="hi
</summary>
<category term="drools" scheme="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/tags/drools/"/>
</entry>
<entry>
<title>HashMap(JDK-1.8)</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/09/05/HashMap-JDK-1-8/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/09/05/HashMap-JDK-1-8/</id>
<published>2018-09-05T07:43:36.000Z</published>
<updated>2018-09-05T07:52:47.298Z</updated>
<content type="html"><![CDATA[<h4 id="一：resize扩容"><a href="#一：resize扩容" class="headerlink" title="一：resize扩容"></a>一：resize扩容</h4><blockquote><ol><li>当前table Node数组赋给临时变量，并记录老table的初始化容量和加载因子</li><li>如果老初始化容量大于0，新表初始化容量直接扩大为原来的2倍</li><li>循环老table，对新table赋值</li></ol></blockquote><blockquote><blockquote><p>1) 如果如果链表只有一个，则进行直接赋值<code>newTab[e.hash & (newCap - 1)] = e</code> </p></blockquote></blockquote><blockquote><blockquote><p>2）如果红黑二叉树：……</p></blockquote></blockquote><blockquote><blockquote><p>3）链表赋值，如果新表位置变化<code>e.hash & oldCap) == 0</code>, 则老表数据在新表位置为（老数组位置i+老数组容量oldCap）</p></blockquote></blockquote><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br></pre></td><td class="code"><pre><span class="line">final Node<K, V>[] resize() {</span><br><span class="line"> Node<K, V>[] oldTab = table;</span><br><span class="line"> int oldCap = (oldTab == null) ? 0 : oldTab.length;// 数组长度</span><br><span class="line"> int oldThr = threshold;// 临界值</span><br><span class="line"> int newCap, newThr = 0;</span><br><span class="line"> if (oldCap > 0) {</span><br><span class="line"> // 扩容</span><br><span class="line"> if (oldCap >= MAXIMUM_CAPACITY) {</span><br><span class="line"> // 原数组长度大于最大容量(1073741824) 则将threshold设为Integer.MAX_VALUE=2147483647</span><br><span class="line"> // 接近MAXIMUM_CAPACITY的两倍</span><br><span class="line"> threshold = Integer.MAX_VALUE;</span><br><span class="line"> return oldTab;</span><br><span class="line"> } else if ((newCap = oldCap << 1) < MAXIMUM_CAPACITY && oldCap >= DEFAULT_INITIAL_CAPACITY) {</span><br><span class="line"> // 新数组长度 是原来的2倍，</span><br><span class="line"> // 临界值也扩大为原来2倍</span><br><span class="line"> newThr = oldThr << 1;</span><br><span class="line"> }</span><br><span class="line"> } else if (oldThr > 0) {</span><br><span class="line"> // 如果原来的thredshold大于0则将容量设为原来的thredshold</span><br><span class="line"> // 在第一次带参数初始化时候会有这种情况</span><br><span class="line"> newCap = oldThr;</span><br><span class="line"> } else {</span><br><span class="line"> // 在默认无参数初始化会有这种情况</span><br><span class="line"> newCap = DEFAULT_INITIAL_CAPACITY;// 16</span><br><span class="line"> newThr = (int) (DEFAULT_LOAD_FACTOR * DEFAULT_INITIAL_CAPACITY);// 0.75*16=12</span><br><span class="line"> }</span><br><span class="line"> if (newThr == 0) {</span><br><span class="line"> // 如果新 的容量 ==0</span><br><span class="line"> float ft = (float) newCap * loadFactor;// loadFactor 哈希加载因子 默认0.75,可在初始化时传入,16*0.75=12 可以放12个键值对</span><br><span class="line"> newThr = (newCap < MAXIMUM_CAPACITY && ft < (float) MAXIMUM_CAPACITY ? (int) ft : Integer.MAX_VALUE);</span><br><span class="line"> }</span><br><span class="line"> threshold = newThr;// 将临界值设置为新临界值</span><br><span class="line"> @SuppressWarnings({ "rawtypes", "unchecked" })</span><br><span class="line"> // 扩容</span><br><span class="line"> Node<K, V>[] newTab = (Node<K, V>[]) new Node[newCap];</span><br><span class="line"> table = newTab;</span><br><span class="line"> // 如果原来的table有数据，则将数据复制到新的table中</span><br><span class="line"> if (oldTab != null) {</span><br><span class="line"> // 根据容量进行循环整个数组，将非空元素进行复制</span><br><span class="line"> for (int j = 0; j < oldCap; ++j) {</span><br><span class="line"> Node<K, V> e;</span><br><span class="line"> // 获取数组的第j个元素</span><br><span class="line"> if ((e = oldTab[j]) != null) {</span><br><span class="line"> oldTab[j] = null;</span><br><span class="line"> // 如果链表只有一个，则进行直接赋值</span><br><span class="line"> if (e.next == null)</span><br><span class="line"> // e.hash & (newCap - 1) 确定元素存放位置</span><br><span class="line"> newTab[e.hash & (newCap - 1)] = e;</span><br><span class="line"> // 此处省略红黑树</span><br><span class="line"> else {</span><br><span class="line"> // 进行链表复制</span><br><span class="line"> // 方法比较特殊： 它并没有重新计算元素在数组中的位置</span><br><span class="line"> // 而是采用了 原始位置加原数组长度的方法计算得到位置</span><br><span class="line"> Node<K, V> loHead = null, loTail = null;</span><br><span class="line"> Node<K, V> hiHead = null, hiTail = null;</span><br><span class="line"> Node<K, V> next;</span><br><span class="line"> do {</span><br><span class="line"> next = e.next;</span><br><span class="line"> // 注意：不是(e.hash & (oldCap-1));而是(e.hash & oldCap)</span><br><span class="line"></span><br><span class="line"> // (e.hash & oldCap) 得到的是 元素的在数组中的位置是否需要移动,示例如下</span><br><span class="line"> // 示例1：</span><br><span class="line"> // e.hash=10 0000 1010</span><br><span class="line"> // oldCap=16 0001 0000</span><br><span class="line"> // & =0 0000 0000 比较高位的第一位 0</span><br><span class="line"> //结论：元素位置在扩容后数组中的位置没有发生改变</span><br><span class="line"></span><br><span class="line"> // 示例2：</span><br><span class="line"> // e.hash=17 0001 0001</span><br><span class="line"> // oldCap=16 0001 0000</span><br><span class="line"> // & =1 0001 0000 比较高位的第一位 1</span><br><span class="line"> //结论：元素位置在扩容后数组中的位置发生了改变，新的下标位置是原下标位置+原数组长度</span><br><span class="line"></span><br><span class="line"> // (e.hash & (oldCap-1)) 得到的是下标位置,示例如下</span><br><span class="line"> // e.hash=10 0000 1010</span><br><span class="line"> // oldCap-1=15 0000 1111</span><br><span class="line"> // & =10 0000 1010</span><br><span class="line"></span><br><span class="line"> // e.hash=17 0001 0001</span><br><span class="line"> // oldCap-1=15 0000 1111</span><br><span class="line"> // & =1 0000 0001</span><br><span class="line"></span><br><span class="line"> //新下标位置</span><br><span class="line"> // e.hash=17 0001 0001</span><br><span class="line"> // newCap-1=31 0001 1111 newCap=32</span><br><span class="line"> // & =17 0001 0001 1+oldCap = 1+16</span><br><span class="line"></span><br><span class="line"> //元素在重新计算hash之后，因为n变为2倍，那么n-1的mask范围在高位多1bit(红色)，因此新的index就会发生这样的变化：</span><br><span class="line"> // 0000 0001->0001 0001</span><br><span class="line"></span><br><span class="line"> if ((e.hash & oldCap) == 0) {</span><br><span class="line"> // 如果原元素位置没有发生变化</span><br><span class="line"> if (loTail == null)</span><br><span class="line"> loHead = e;// 确定首元素</span><br><span class="line"> // 第一次进入时 e -> aa ; loHead-> aa</span><br><span class="line"> else</span><br><span class="line"> loTail.next = e;</span><br><span class="line"> //第二次进入时 loTail-> aa ; e -> bb ; loTail.next -> bb;而loHead和loTail是指向同一块内存的，所以loHead.next 地址为 bb </span><br><span class="line"> //第三次进入时 loTail-> bb ; e -> cc ; loTail.next 地址为 cc;loHead.next.next = cc</span><br><span class="line"> loTail = e;</span><br><span class="line"> // 第一次进入时 e -> aa ; loTail-> aa loTail指向了和 loHead相同的内存空间</span><br><span class="line"> // 第二次进入时 e -> bb ; loTail-> bb loTail指向了和 loTail.next（loHead.next）相同的内存空间 loTail=loTail.next</span><br><span class="line"> // 第三次进入时 e -> cc ; loTail-> cc loTail指向了和 loTail.next(loHead.next.next)相同的内存</span><br><span class="line"> } else {</span><br><span class="line"> //与上面同理</span><br><span class="line"></span><br><span class="line"> if (hiTail == null)</span><br><span class="line"> hiHead = e;</span><br><span class="line"> else</span><br><span class="line"> hiTail.next = e;</span><br><span class="line"> hiTail = e;</span><br><span class="line"> }</span><br><span class="line"> } while ((e = next) != null);//这一块就是 旧链表迁移新链表</span><br><span class="line"> //总结：1.8中 旧链表迁移新链表 链表元素相对位置没有变化; 实际是对对象的内存地址进行操作 </span><br><span class="line"> //在1.7中 旧链表迁移新链表 如果在新表的数组索引位置相同，则链表元素会倒置</span><br><span class="line"> if (loTail != null) {</span><br><span class="line"> loTail.next = null;// 将链表的尾节点 的next 设置为空</span><br><span class="line"> newTab[j] = loHead;</span><br><span class="line"> }</span><br><span class="line"> if (hiTail != null) {</span><br><span class="line"> hiTail.next = null;// 将链表的尾节点 的next 设置为空</span><br><span class="line"> newTab[j + oldCap] = hiHead;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> return newTab;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure><h4 id="二、线程安全"><a href="#二、线程安全" class="headerlink" title="二、线程安全"></a>二、线程安全</h4><p>JDK-1.7 线程不安全</p><p>resize()导致线程不安全</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">transfer()函数逻辑</span><br><span class="line">1. 对索引数组中的元素遍历</span><br><span class="line">2. 对链表上的每一个节点遍历：用 next 取得要转移那个元素的下一个，将 e 转移到新 Hash 表的头部，使用头插法插入节点。</span><br><span class="line">3. 循环2，直到链表节点全部转移</span><br><span class="line">4. 循环1，直到所有索引数组全部转移</span><br><span class="line"></span><br><span class="line">经过这几步，我们会发现转移的时候是逆序的。假如转移前链表顺序是1->2->3，那么转移后就会变成3->2->1。这时候就有点头绪了，死锁问题不就是因为1->2的同时2->1造成的吗？所以，HashMap 的死锁问题就出在这个transfer()函数上。</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<h4 id="一：resize扩容"><a href="#一：resize扩容" class="headerlink" title="一：resize扩容"></a>一：resize扩容</h4><blockquote>
<ol>
<li>当前table Node数组赋给临时变
</summary>
<category term="HashMap" scheme="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/tags/HashMap/"/>
</entry>
<entry>
<title>Mockito单元测试</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/09/05/Mockito%E5%8D%95%E5%85%83%E6%B5%8B%E8%AF%95/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/09/05/Mockito单元测试/</id>
<published>2018-09-05T07:42:29.000Z</published>
<updated>2018-09-05T07:52:37.442Z</updated>
<content type="html"><![CDATA[<h4 id="Maven依赖"><a href="#Maven依赖" class="headerlink" title="Maven依赖"></a>Maven依赖</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line"><dependency></span><br><span class="line"> <groupId>junit</groupId></span><br><span class="line"> <artifactId>junit</artifactId></span><br><span class="line"> <version>4.11</version></span><br><span class="line"> <scope>test</scope></span><br><span class="line"></dependency></span><br><span class="line"><dependency></span><br><span class="line"> <groupId>org.mockito</groupId></span><br><span class="line"> <artifactId>mockito-all</artifactId></span><br><span class="line"> <version>2.0.2-beta</version></span><br><span class="line"> <scope>test</scope></span><br><span class="line"></dependency></span><br><span class="line"><dependency></span><br><span class="line"> <groupId>org.powermock</groupId></span><br><span class="line"> <artifactId>powermock-module-junit4</artifactId></span><br><span class="line"> <version>1.6.5</version></span><br><span class="line"> <scope>test</scope></span><br><span class="line"></dependency></span><br><span class="line"><dependency></span><br><span class="line"> <groupId>org.powermock</groupId></span><br><span class="line"> <artifactId>powermock-api-mockito</artifactId></span><br><span class="line"> <version>1.6.5</version></span><br><span class="line"> <scope>test</scope></span><br><span class="line"></dependency></span><br></pre></td></tr></table></figure><h4 id="普通类mock"><a href="#普通类mock" class="headerlink" title="普通类mock"></a>普通类mock</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br></pre></td><td class="code"><pre><span class="line">import org.junit.Assert;</span><br><span class="line">import org.junit.Before;</span><br><span class="line">import org.junit.Test;</span><br><span class="line">import org.mockito.InjectMocks;</span><br><span class="line">import org.mockito.Mock;</span><br><span class="line">import org.mockito.Mockito;</span><br><span class="line">import org.mockito.MockitoAnnotations;</span><br><span class="line">import org.mockito.invocation.InvocationOnMock;</span><br><span class="line">import org.mockito.stubbing.Answer;</span><br><span class="line"></span><br><span class="line">import javax.servlet.http.HttpServletRequest;</span><br><span class="line">import javax.servlet.http.HttpServletResponse;</span><br><span class="line">import java.io.File;</span><br><span class="line">import java.io.PrintWriter;</span><br><span class="line"></span><br><span class="line">public class DataHandlerTest {</span><br><span class="line"></span><br><span class="line"> @InjectMocks</span><br><span class="line"> DataHandler dataHandler = new DataHandler();</span><br><span class="line"></span><br><span class="line"> @Mock</span><br><span class="line"> protected HttpServletRequest request;</span><br><span class="line"></span><br><span class="line"> @Mock</span><br><span class="line"> protected HttpServletResponse response;</span><br><span class="line"></span><br><span class="line"> @Before</span><br><span class="line"> public void before() throws Exception {</span><br><span class="line"> MockitoAnnotations.initMocks(this);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> @Test</span><br><span class="line"> public void testPrintString() throws Exception {</span><br><span class="line"> JsonResult result = new JsonResult();</span><br><span class="line"> Mockito.doNothing().doAnswer(new Answer() {</span><br><span class="line"></span><br><span class="line"> @Override</span><br><span class="line"> public Object answer(InvocationOnMock invocation) throws Throwable {</span><br><span class="line"> return null;</span><br><span class="line"> }</span><br><span class="line"> }).when(response).setHeader("Content-Type", "application/json");</span><br><span class="line"></span><br><span class="line"> String path = DataHandlerTest.class.getResource("/").getPath() + "dataHandler.txt";</span><br><span class="line"> File file = new File(path);</span><br><span class="line"> if (!file.exists()) {</span><br><span class="line"> file.createNewFile();</span><br><span class="line"> }</span><br><span class="line"> PrintWriter writer = new PrintWriter(file);</span><br><span class="line"> Mockito.when(response.getWriter()).thenReturn(writer);</span><br><span class="line"> dataHandler.print(result);</span><br><span class="line"></span><br><span class="line"> if (file != null && file.exists()) {</span><br><span class="line"> file.delete();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h4 id="静态类mock"><a href="#静态类mock" class="headerlink" title="静态类mock"></a>静态类mock</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br></pre></td><td class="code"><pre><span class="line">@RunWith(PowerMockRunner.class)</span><br><span class="line">@PrepareForTest(ThreadContext.class)</span><br><span class="line">public class CartTest {</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> @InjectMocks</span><br><span class="line"> private Cart cart = new Cart();</span><br><span class="line"></span><br><span class="line"> @Mock</span><br><span class="line"> ThreadContext threadContext;</span><br><span class="line"></span><br><span class="line"> @Mock</span><br><span class="line"> protected HttpServletRequest request;</span><br><span class="line"> </span><br><span class="line"> @Mock</span><br><span class="line"> protected HttpServletResponse response;</span><br><span class="line"></span><br><span class="line"> @Before</span><br><span class="line"> public void before() throws Exception { </span><br><span class="line"> MockitoAnnotations.initMocks(this);</span><br><span class="line"> PowerMockito.mockStatic(ThreadContext.class);</span><br><span class="line"></span><br><span class="line"> ThreadContext threadContext2 = new ThreadContext();</span><br><span class="line"> threadContext2.setLoginId("123");</span><br><span class="line"></span><br><span class="line"> //Mock当前用户信息 currentUser</span><br><span class="line"> Mockito.when(ThreadContext.getCurrentUser()).thenReturn(threadContext2);</span><br><span class="line"></span><br><span class="line"> //Mock response.setHeader方法</span><br><span class="line"> Mockito.doAnswer(new Answer<Object>() {</span><br><span class="line"> public Object answer(InvocationOnMock invocation) {</span><br><span class="line"> Object[] args = invocation.getArguments();</span><br><span class="line"> return "called with arguments: " + args;</span><br><span class="line"> }</span><br><span class="line"> }).when(response).setHeader(Mockito.anyString(),Mockito.anyString());</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> </span><br><span class="line"> /** </span><br><span class="line"> * </span><br><span class="line"> * Method: screen(Context context) </span><br><span class="line"> * </span><br><span class="line"> */ </span><br><span class="line"> @Test</span><br><span class="line"> public void test() throws Exception {</span><br><span class="line"> Map<String, String> map = Maps.newHashMap();</span><br><span class="line"> map.put("Login","test");</span><br><span class="line"> Context context = new MappedContext();</span><br><span class="line"></span><br><span class="line"> //Mock request中获取回调函数名的方法</span><br><span class="line"> Mockito.when(request.getParameter(Mockito.any(String.class))).thenReturn("callback");</span><br><span class="line"> cart.screen(context);</span><br><span class="line"></span><br><span class="line"> }</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<h4 id="Maven依赖"><a href="#Maven依赖" class="headerlink" title="Maven依赖"></a>Maven依赖</h4><figure class="highlight plain"><table><tr><td class=
</summary>
<category term="mockito" scheme="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/tags/mockito/"/>
</entry>
<entry>
<title>MySQL事务级别</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/09/05/MySQL%E4%BA%8B%E5%8A%A1%E7%BA%A7%E5%88%AB/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/09/05/MySQL事务级别/</id>
<published>2018-09-05T07:41:29.000Z</published>
<updated>2018-09-05T07:52:28.978Z</updated>
<content type="html"><![CDATA[<p>幻读：事务1读取数据时，事务2增加并提交，事务1再次读取数据时，可以看到事务B新增的数据。</p><p>不可重复读：事务1读取数据时，事务2修改并提交该数据，事务1再次读取事务时，可以看到事务B修改后的记录</p><p>脏读：事务1更新了记录，但没有提交，事务2读取了更新后的行，然后事务T1回滚，现在T2读取无效。</p><h4 id="事务级别"><a href="#事务级别" class="headerlink" title="事务级别"></a>事务级别</h4><ol><li>read uncommit：可能出现脏读，幻读，不可重复读</li><li>read commit : 不允许出现脏读，可能出现幻读，不可重复读</li><li>repeaable commit : 不会出现脏读、不可重复读，可能出现幻读</li><li>seralizeable : 不会出现脏读，幻读，不可重复读</li></ol><p><a href="http://www.cnblogs.com/hustcat/archive/2009/10/28/1591648.html" target="_blank" rel="noopener">http://www.cnblogs.com/hustcat/archive/2009/10/28/1591648.html</a></p>]]></content>
<summary type="html">
<p>幻读：事务1读取数据时，事务2增加并提交，事务1再次读取数据时，可以看到事务B新增的数据。</p>
<p>不可重复读：事务1读取数据时，事务2修改并提交该数据，事务1再次读取事务时，可以看到事务B修改后的记录</p>
<p>脏读：事务1更新了记录，但没有提交，事务2读取了更
</summary>
<category term="mysql" scheme="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/tags/mysql/"/>
</entry>
<entry>
<title>spring schema扩展</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/09/05/spring-schema%E6%89%A9%E5%B1%95/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/09/05/spring-schema扩展/</id>
<published>2018-09-05T07:39:49.000Z</published>
<updated>2018-09-05T07:52:18.692Z</updated>
<content type="html">< , 如下`people.xsd`文件:</span><br><span class="line"></span><br><span class="line">```xml</span><br><span class="line"><?xml version="1.0" encoding="UTF-8"?></span><br><span class="line"><xsd:schema xmlns="http://www.pomelo.com/schema/people"</span><br><span class="line"> xmlns:xsd="http://www.w3.org/2001/XMLSchema"</span><br><span class="line"> xmlns:beans="http://www.springframework.org/schema/beans"</span><br><span class="line"> targetNamespace="http://www.pomelo.com/schema/people"</span><br><span class="line"> elementFormDefault="qualified"</span><br><span class="line"> attributeFormDefault="unqualified"></span><br><span class="line"> <xsd:import namespace="http://www.springframework.org/schema/beans"/></span><br><span class="line"></span><br><span class="line"> <xsd:element name="student"></span><br><span class="line"> <xsd:complexType></span><br><span class="line"> <xsd:complexContent></span><br><span class="line"> <xsd:extension base="beans:identifiedType"></span><br><span class="line"></span><br><span class="line"> <xsd:attribute name="name" type="xsd:string"></span><br><span class="line"> <xsd:annotation></span><br><span class="line"> <xsd:documentation>姓名</xsd:documentation></span><br><span class="line"> </xsd:annotation></span><br><span class="line"> </xsd:attribute></span><br><span class="line"></span><br><span class="line"> <xsd:attribute name="age" type="xsd:string"></span><br><span class="line"> <xsd:annotation></span><br><span class="line"> <xsd:documentation>年龄</xsd:documentation></span><br><span class="line"> </xsd:annotation></span><br><span class="line"> </xsd:attribute></span><br><span class="line"></span><br><span class="line"> </xsd:extension></span><br><span class="line"> </xsd:complexContent></span><br><span class="line"> </xsd:complexType></span><br><span class="line"> </xsd:element></span><br><span class="line"></xsd:schema></span><br></pre></td></tr></table></figure><h4 id="二、编写自定义NamespaceHandler"><a href="#二、编写自定义NamespaceHandler" class="headerlink" title="二、编写自定义NamespaceHandler"></a>二、编写自定义NamespaceHandler</h4><figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> schema;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> org.springframework.beans.factory.xml.NamespaceHandlerSupport;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">StudentNamespaceHandler</span> <span class="keyword">extends</span> <span class="title">NamespaceHandlerSupport</span> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">init</span><span class="params">()</span> </span>{</span><br><span class="line"> registerBeanDefinitionParser(<span class="string">"student"</span>, <span class="keyword">new</span> StudentBeanDefinitionParser());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure><h4 id="三、编写BeanDefinition"><a href="#三、编写BeanDefinition" class="headerlink" title="三、编写BeanDefinition"></a>三、编写BeanDefinition</h4><figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> schema;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> org.springframework.beans.factory.support.BeanDefinitionBuilder;</span><br><span class="line"><span class="keyword">import</span> org.springframework.beans.factory.xml.AbstractSingleBeanDefinitionParser;</span><br><span class="line"><span class="keyword">import</span> org.springframework.util.StringUtils;</span><br><span class="line"><span class="keyword">import</span> org.w3c.dom.Element;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">StudentBeanDefinitionParser</span> <span class="keyword">extends</span> <span class="title">AbstractSingleBeanDefinitionParser</span> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">protected</span> Class <span class="title">getBeanClass</span><span class="params">(Element element)</span> </span>{</span><br><span class="line"> <span class="keyword">return</span> Student.class;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title">doParse</span><span class="params">(Element element, BeanDefinitionBuilder bean)</span> </span>{</span><br><span class="line"> String name = element.getAttribute(<span class="string">"name"</span>);</span><br><span class="line"> bean.addPropertyValue(<span class="string">"name"</span>, name);</span><br><span class="line"></span><br><span class="line"> String age = element.getAttribute(<span class="string">"age"</span>);</span><br><span class="line"> <span class="keyword">if</span> (StringUtils.hasText(age)) {</span><br><span class="line"> bean.addPropertyValue(<span class="string">"age"</span>, Integer.valueOf(age));</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>实体类：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br></pre></td><td class="code"><pre><span class="line">package schema;</span><br><span class="line"></span><br><span class="line">public class Student {</span><br><span class="line"> </span><br><span class="line"> private String name; </span><br><span class="line"> </span><br><span class="line"> private int age; </span><br><span class="line"> </span><br><span class="line"> public String getName() { </span><br><span class="line"> return name; </span><br><span class="line"> } </span><br><span class="line"> </span><br><span class="line"> public void setName(String name) { </span><br><span class="line"> this.name = name; </span><br><span class="line"> } </span><br><span class="line"> </span><br><span class="line"> public int getAge() { </span><br><span class="line"> return age; </span><br><span class="line"> } </span><br><span class="line"> </span><br><span class="line"> public void setAge(int age) { </span><br><span class="line"> this.age = age; </span><br><span class="line"> } </span><br><span class="line"> </span><br><span class="line">}</span><br></pre></td></tr></table></figure><h4 id="四、注册schema组件"><a href="#四、注册schema组件" class="headerlink" title="四、注册schema组件"></a>四、注册schema组件</h4><p>最后在META-INF目录下添加两个配置文件(<code>spring.handler</code>和<code>spring.schema</code>):</p><p><code>spring.handler</code>配置如下：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">http\://www.pomelo.com/schema/people=schema.StudentNamespaceHandler</span><br></pre></td></tr></table></figure><p><code>spring.schema</code>配置如下：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">http\://www.pomelo.com/schema/people.xsd=META-INF/people.xsd</span><br></pre></td></tr></table></figure><h4 id="五、测试"><a href="#五、测试" class="headerlink" title="五、测试"></a>五、测试</h4><p>新建<code>applicationContext.xml</code>放在clasapath下面：</p><figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><?xml version="1.0" encoding="UTF-8"?></span><br><span class="line"><span class="tag"><<span class="name">beans</span> <span class="attr">xmlns</span>=<span class="string">"http://www.springframework.org/schema/beans"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">xmlns:xsi</span>=<span class="string">"http://www.w3.org/2001/XMLSchema-instance"</span> <span class="attr">xmlns:people</span>=<span class="string">"http://www.pomelo.com/schema/people"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">xsi:schemaLocation</span>=<span class="string">"http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.pomelo.com/schema/people http://www.pomelo.com/schema/people.xsd"</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="tag"><<span class="name">people:student</span> <span class="attr">id</span>=<span class="string">"student1"</span> <span class="attr">name</span>=<span class="string">"student1"</span> <span class="attr">age</span>=<span class="string">"18"</span>/></span></span><br><span class="line"></span><br><span class="line"> <span class="tag"><<span class="name">people:student</span> <span class="attr">id</span>=<span class="string">"student2"</span> <span class="attr">name</span>=<span class="string">"student2"</span> <span class="attr">age</span>=<span class="string">"20"</span> /></span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="tag"><<span class="name">bean</span> <span class="attr">id</span>=<span class="string">"student3"</span> <span class="attr">class</span>=<span class="string">"schema.Student"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"name"</span> <span class="attr">value</span>=<span class="string">"student3"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"age"</span> <span class="attr">value</span>=<span class="string">"23"</span>/></span></span><br><span class="line"> <span class="tag"></<span class="name">bean</span>></span></span><br><span class="line"></span><br><span class="line"><span class="tag"></<span class="name">beans</span>></span></span><br></pre></td></tr></table></figure><p>java调用：</p><figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> schema;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> org.springframework.context.ApplicationContext;</span><br><span class="line"><span class="keyword">import</span> org.springframework.context.support.ClassPathXmlApplicationContext;</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Created by zhengyong on 17/3/3.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">SchemaTest</span> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">main</span><span class="params">(String[] args)</span> </span>{</span><br><span class="line"></span><br><span class="line"> ApplicationContext ctx = <span class="keyword">new</span> ClassPathXmlApplicationContext(<span class="string">"/applicationContext.xml"</span>);</span><br><span class="line"> Student student1 = (Student) ctx.getBean(<span class="string">"student1"</span>);</span><br><span class="line"> Student student2 = (Student) ctx.getBean(<span class="string">"student2"</span>);</span><br><span class="line"> Student student3 = (Student) ctx.getBean(<span class="string">"student3"</span>);</span><br><span class="line"></span><br><span class="line"> System.out.println(<span class="string">"name: "</span> + student1.getName() + <span class="string">" age :"</span> + student1.getAge());</span><br><span class="line"> System.out.println(<span class="string">"name: "</span> + student2.getName() + <span class="string">" age :"</span> + student2.getAge());</span><br><span class="line"> System.out.println(<span class="string">"name: "</span> + student3.getName() + <span class="string">" age :"</span> + student3.getAge());</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>具体代码详见：<a href="https://github.com/zyongjava/pomelo/blob/master/src/main/resources/META-INF/people.xsd" target="_blank" rel="noopener">https://github.com/zyongjava/pomelo/blob/master/src/main/resources/META-INF/people.xsd</a></p>]]></content>
<summary type="html">
<h3 id="介绍"><a href="#介绍" class="headerlink" title="介绍"></a>介绍</h3><blockquote>
<p>Spring 2.5在2.0的基于Schema的Bean配置的基础之上，再增加了扩展XML配置的机制。通过该机制，
</summary>
<category term="schema" scheme="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/tags/schema/"/>
</entry>
<entry>
<title>swagger自动生成API文档</title>
<link href="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/09/05/swagger%E8%87%AA%E5%8A%A8%E7%94%9F%E6%88%90API%E6%96%87%E6%A1%A3/"/>
<id>https://blog.csdn.net/zhengyong15984285623?viewmode=contents/2018/09/05/swagger自动生成API文档/</id>
<published>2018-09-05T07:38:43.000Z</published>
<updated>2018-09-05T07:39:09.913Z</updated>
<content type="html"><![CDATA[<h4 id="一、pom-xml配置"><a href="#一、pom-xml配置" class="headerlink" title="一、pom.xml配置"></a>一、pom.xml配置</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><!--swagger --></span><br><span class="line"><dependency></span><br><span class="line"> <groupId>io.springfox</groupId></span><br><span class="line"> <artifactId>springfox-swagger2</artifactId></span><br><span class="line"> <version>2.8.0</version></span><br><span class="line"></dependency></span><br><span class="line"><dependency></span><br><span class="line"> <groupId>io.springfox</groupId></span><br><span class="line"> <artifactId>springfox-swagger-ui</artifactId></span><br><span class="line"> <version>2.8.0</version></span><br><span class="line"></dependency></span><br></pre></td></tr></table></figure><p>二、spring boot配置</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br></pre></td><td class="code"><pre><span class="line">package cn.pomelo.web.config;</span><br><span class="line"></span><br><span class="line">import org.springframework.beans.factory.annotation.Value;</span><br><span class="line">import org.springframework.context.annotation.Bean;</span><br><span class="line">import org.springframework.context.annotation.Configuration;</span><br><span class="line">import springfox.documentation.builders.ApiInfoBuilder;</span><br><span class="line">import springfox.documentation.builders.PathSelectors;</span><br><span class="line">import springfox.documentation.builders.RequestHandlerSelectors;</span><br><span class="line">import springfox.documentation.service.ApiInfo;</span><br><span class="line">import springfox.documentation.service.Contact;</span><br><span class="line">import springfox.documentation.spi.DocumentationType;</span><br><span class="line">import springfox.documentation.spring.web.plugins.Docket;</span><br><span class="line">import springfox.documentation.swagger2.annotations.EnableSwagger2;</span><br><span class="line"></span><br><span class="line">/**</span><br><span class="line"> * swagger自动生成API文档 (http://127.0.0.1:8080/swagger-ui.html)</span><br><span class="line"> */</span><br><span class="line">@EnableSwagger2</span><br><span class="line">@Configuration</span><br><span class="line">public class SwaggerConfig {</span><br><span class="line"> </span><br><span class="line"> //是否开启swagger，正式环境一般是需要关闭的，可根据springboot的多环境配置进行设置</span><br><span class="line"> @Value(value = "${swagger.enabled}")</span><br><span class="line"> Boolean swaggerEnabled;</span><br><span class="line"> </span><br><span class="line"> @Bean</span><br><span class="line"> public Docket createRestApi() {</span><br><span class="line"> return new Docket(DocumentationType.SWAGGER_2).apiInfo(apiInfo())</span><br><span class="line"> // 是否开启</span><br><span class="line"> .enable(swaggerEnabled).select()</span><br><span class="line"> // 扫描的路径包</span><br><span class="line"> .apis(RequestHandlerSelectors.basePackage("cn.pomelo.web.controller"))</span><br><span class="line"> // 指定路径处理PathSelectors.any()代表所有的路径</span><br><span class="line"> .paths(PathSelectors.any()).build().pathMapping("/");</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> private ApiInfo apiInfo() {</span><br><span class="line"> return new ApiInfoBuilder()</span><br><span class="line"> .title("SpringBoot-Swagger2集成和使用-demo示例")</span><br><span class="line"> .description("spring boot swagger restful api document")</span><br><span class="line"> // 作者信息</span><br><span class="line"> .contact(new Contact("yong.zheng", "https://blog.csdn.net/zhengyong15984285623?viewmode=contents", "zyongjava@163.com"))</span><br><span class="line"> .version("1.0.0")</span><br><span class="line"> .build();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>三、swagger-ui.html配置</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line">import org.springframework.context.annotation.Configuration;</span><br><span class="line">import org.springframework.web.servlet.config.annotation.ResourceHandlerRegistry;</span><br><span class="line">import org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter;</span><br><span class="line"></span><br><span class="line">/**</span><br><span class="line"> * MVC配置</span><br><span class="line"> *</span><br><span class="line"> */</span><br><span class="line">@Configuration</span><br><span class="line">public class CustomMvcConfig extends WebMvcConfigurerAdapter {</span><br><span class="line"> @Override</span><br><span class="line"> public void addResourceHandlers(ResourceHandlerRegistry registry) {</span><br><span class="line"> super.addResourceHandlers(registry);</span><br><span class="line"> registry.addResourceHandler("swagger-ui.html")</span><br><span class="line"> .addResourceLocations("classpath:/META-INF/resources/");</span><br><span class="line"></span><br><span class="line"> registry.addResourceHandler("/webjars/**")</span><br><span class="line"> .addResourceLocations("classpath:/META-INF/resources/webjars/");</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>四、注解说明</p><p>常用到的注解：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">@Api</span><br><span class="line">@ApiModel</span><br><span class="line">@ApiModelProperty</span><br><span class="line">@ApiOperation</span><br><span class="line">@ApiParam</span><br><span class="line">@ApiResponse</span><br><span class="line">@ApiResponses</span><br><span class="line">@ResponseHeader</span><br></pre></td></tr></table></figure><p>地址：<a href="https://www.cnblogs.com/hyl8218/p/8520994.html" target="_blank" rel="noopener">https://www.cnblogs.com/hyl8218/p/8520994.html</a></p>]]></content>
<summary type="html">
<h4 id="一、pom-xml配置"><a href="#一、pom-xml配置" class="headerlink" title="一、pom.xml配置"></a>一、pom.xml配置</h4><figure class="highlight plain"><tabl
</summary>
<category term="swagger" scheme="https://blog.csdn.net/zhengyong15984285623?viewmode=contents/tags/swagger/"/>
</entry>
</feed>