行首行末添加字符串
实际应用场景:
	我们现在有一个只有cmd功能的system权限的windows shell
	主机为xp 32位,3389处于打开状态,我们想使用ew将其3389端口代理出来,但是xp既没有bitsadmin又没有certutil,我们可以选择使用vbs脚本进行下载,这时我们就需要通过echo命令来将vbs脚本的内容写入到目标主机的一个文件中,如果手动一行一行地使用echo命令写入,实在是太麻烦了,我们可以借助sed强大的文本处理功能,在原来的vbs的每一行的行首和行尾加上“echo ”和“>>%temp\update.vbs%”,首先将vbs内容写到test中,然后执行下面两条命令,复制test中的内容,直接粘贴到我们的cmd shell上,一次性执行完所有的写入命令,最后再执行cscript.exe %temp%\update.vbs即可
	
		sed -i 's/^/&echo /g' test
		sed -i 's/$/& >>%temp%\\update.vbs/g' test
		
	第二条命令中多出来的反斜杠为转义符
搜索当前目录中的所有文件,查找指定字符串

​ 应用场景,调试PHP时,寻找某一字符串是由哪一个文件输出的

	grep -nr "print" *
		r选项代表递归(查询子目录)
		*代表所有文件
截取字符串

应用场景:
​ 现在我要从一份泄露的邮箱账户中截取出密码,形式如下:
test@test.com:test

​ 我需要截取出:后面的字符串

只需要使用下面这条命令即可:

var="test@test.com:test"
echo ${var#*:}
获取子串在母串中的偏移量

echo $a | grep -bo ":" | sed 's/:.*$//'

grep -bo ":"打印样式匹配所位于的字符或字节偏移,如果不使用sed进行处理,会把我们要匹配的字符也跟在后面,并使用:分割,因此我们需要使用sed的正则匹配,将:以及:之后的所有字符替换成空

反转字符串

echo str | rev

剔除含有某一字符串的行

sed -i '/keywords/d' filename

提取出包含某一字符串的行

sed -n '/keywords/p' filename

JS提取域中的主机

我们使用dsquery导出的计算机结果基本上是对下面这个样子:

"CN=CR-50,OU=Coumputers_CR,DC=domain,DC=com"
"CN=CR-55,OU=Coumputers_CR,DC=domain,DC=com"
"CN=CR-54,OU=Coumputers_CR,DC=domain,DC=com"

我们可以使用下面这个脚本,将主机名提取出来:

var str = `
"CN=CR-50,OU=Coumputers_CR,DC=domain,DC=com"
"CN=CR-55,OU=Coumputers_CR,DC=domain,DC=com"
"CN=CR-54,OU=Coumputers_CR,DC=domain,DC=com"
`


str.trim().split('\n').forEach(function(v, i) {
var result = /.*?"CN=(.*?),.*?/.exec(v);
document.write(result[1]);
document.write("</br>");
})

这样就可以在浏览器中输出主机名了

多种方式文本处理——javascript脚本、bash脚本、python脚本_经验分享

对脚本的改进,这样的正则可以防止匹配到dn中的其他OU,只筛选出dn中带有OU=TTT的dn中的主机名部分,因为我们使用[\w|-]*?,排除了,的出现,这样就不会匹配到其他的OU,又因为是(非贪婪)模式,所以就达到了预期的目的

str.trim().split('\n').forEach(function(v, i) {
var result = /.*?CN=([\w|-]*?),.*?OU=TTT.*?/.exec(v);
if(result !== null) {
document.write(result[1]+".megafon.ru");
document.write("</br>");
}
})

然后用该脚本测试那些主机可以联通:

@echo off

setlocal ENABLEDELAYEDEXPANSION 

for /f %%i in (dn.txt) do (
	echo %%i
	net use \\%%i\ipc$ "password" /user:cotoso.com\administrator >>temp.tmp 2>&1
	for /f %%l in ('type temp.tmp ^| find /i "1326" /c') do ( 
        if %%l equ 1 (
			echo %%i>>res.txt )
	)
	del temp.tmp
)
使用JS从任意字符串中提取出IP地址

正则表达式:

(\d{1,3}\.){3}\d{1,3}

\d{1,3}\.表示匹配一个1~3个数字组成的字符串和一个.

d.元字符需要转义所有前面的那个\,这个是正则的基础语法,不需要我多解释

将它们用()括起来,作为一个匹配模式,后面的{3}表示该匹配模式要重复3次,最后再跟上一个\d{1,3},由1~3个字符组成的数字

这样就能匹配到所有符合IP地址特征的字符串了

使用JS实现,代码如下:

var str = `
172.96.244.114  testsetsetset   testsetsetset    testsetsetset  testsetsetset104.224.181.204     asdasdasd   testsetsetset    testsetsetset  testsetsetset67.230.185.196  testsetsetset   testsetsetset    testsetsetset    testsetsetset104.224.144.160     testsetsetset   testsetsetset    asdasdasd                    testsetsetset104.224.182.250     testsetsetset   testsetsetset    testsetsetset             testsetsetset104.224.190.79  testsetsetset testsetsetset     2oixMlMlzAPPkUxt    testsetsetset   testsetsetset95.163.207.244  testsetsetset   testsetsetset    syi71eRZiehv                    testsetsetset172.96.241.42   testsetsetset   testsetsetset    testsetsetset  testsetsetset198.181.40.163  testsetsetset   testsetsetset    testsetsetset  testsetsetset104.224.178.144     testsetsetset   testsetsetset    testsetsetset  testsetsetset104.224.177.150     testsetsetset   testsetsetset    testsetsetset  testsetsetsetasdasdas
`
var result = str.match(/(\d{1,3}\.){3}\d{1,3}/g);
for ( var i = 0; i <result.length; i++){
	document.write(result[i]);
	document.write("<br />");
}
IP 排序

sort -t . -k 3,3n -k 4,4n /path/to/file >> /path/to/newfile

JAvascript正则提取出符合条件的字符串
<html>
    <head>
        <link rel="stylesheet" href="http://code.jquery.com/ui/1.11.3/themes/smoothness/jquery-ui.css">
        <script src="http://code.jquery.com/jquery-1.10.2.js"></script>
        <script src="http://code.jquery.com/ui/1.11.3/jquery-ui.js"></script>
    </head>
    <body>
        
            function PreviewText() {
				var oFReader = new FileReader();
				oFReader.readAsDataURL(document.getElementById("uploadText").files[0]);
				oFReader.onload = function (oFREvent) {
					document.getElementById("uploadTextValue").value = oFREvent.target.result; 
					//oFREvent.target.result);
				var decodedString = atob(oFREvent.target.result.split("base64,")[1]);
					//data:text/html;base64,
					//data:text/plain;base64,
					//document.getElementsByTagName('xmp')[0].innerHTML = decodedString;//oFREvent.target.result;//;
					
					var lines = decodedString.split('\n');
					var fruits = [];
					for(var i = 0;i < lines.length;i++){
						var result = /dn: DC=(.*?),DC=corp.contoso.com,/.exec(lines[i])

						if(result != null)
							fruits.push(result[1])
					}
					//排序
					fruits.sort();
					
						
						//去重
					var uniqueNames = [];
					$.each(fruits, function(i, el){
						if($.inArray(el, uniqueNames) === -1) uniqueNames.push(el);
					});
					//输出
					for(var i = 0;i < uniqueNames.length;i++)
						
						document.write(uniqueNames[i] + "<br>");
				};
			};
        jQuery(document).ready(function(){
            $('#viewSource').click(function ()
            {
                var text = $('#uploadTextValue').val();
                alert(text);
                //here ajax
            });
        });
        </script>
        <xmp></xmp>
        <div>
            <input type="hidden" id="uploadTextValue" name="uploadTextValue" value="" />
            <input id="uploadText" style="width:120px" type="file" size="10"  onchange="PreviewText();" />
        </div>
        <a href="#" id="viewSource">Source file</a>
    </body>
</html>


javascript删除vbs源文件中的注释行
<html>
    <head>
        <link rel="stylesheet" href="http://code.jquery.com/ui/1.11.3/themes/smoothness/jquery-ui.css">
        <script src="http://code.jquery.com/jquery-1.10.2.js"></script>
        <script src="http://code.jquery.com/ui/1.11.3/jquery-ui.js"></script>
    </head>
    <body>
        <script>
            function PreviewText() {
				var oFReader = new FileReader();
				oFReader.readAsDataURL(document.getElementById("uploadText").files[0]);
				oFReader.onload = function (oFREvent) {
					document.getElementById("uploadTextValue").value = oFREvent.target.result; 
					//oFREvent.target.result);
				var decodedString = atob(oFREvent.target.result.split("base64,")[1]);


                var string = "";
                var lines = decodedString.split('\n');
					var fruits = [];
					for(var i = 0;i < lines.length;i++){
						
				
 var temp = lines[i];
 
//去掉所有的空格(中文空格、英文空格都会被替换)
temp = temp.replace(/\s/g,"");
if(temp.indexOf("'")!=0)
//输出转换后的字符串
string = string+ lines[i];
                    }
                    document.getElementsByTagName('xmp')[0].innerHTML=string;
			    };
            }
        jQuery(document).ready(function(){
            $('#viewSource').click(function ()
            {
                var text = $('#uploadTextValue').val();
                alert(text);
                //here ajax
            });
        });
        </script>
        <xmp></xmp>
        <div>
            <input type="hidden" id="uploadTextValue" name="uploadTextValue" value="" />
            <input id="uploadText" style="width:120px" type="file" size="10"  onchange="PreviewText();" />
        </div>
        <a href="#" id="viewSource">Source file</a>
    </body>
</html>
对nmap结果进行处理
<script type="text/javascript">
var str = `
Nmap scan report for 10.50.195.1
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.50.210.35
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.50.210.40
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.50.211.25
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.50.245.2
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.52.54.239
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.52.240.134
Host is up (0.44s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.52.240.22
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.52.240.6
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.52.240.7
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.54.123.6
Host is up (0.46s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.54.123.7
Host is up (0.48s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.54.123.70
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.54.123.76
Host is up (0.46s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.54.124.7
Host is up (0.48s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.54.124.9
Host is up (0.46s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.54.163.155
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.54.163.157
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.55.167.85
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.59.5.208
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.61.2.128
Host is up (0.46s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.61.2.147
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.61.9.196
Host is up (0.45s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.61.9.197
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.61.9.203
Host is up (0.47s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.63.158.6
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.63.158.7
Host is up (0.42s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.66.40.132
Host is up (0.45s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.66.40.134
Host is up (0.44s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.66.40.144
Host is up (0.45s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.66.40.250
Host is up (0.45s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.75.67.12
Host is up (0.52s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.75.67.13
Host is up (0.51s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.78.122.90
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.60.59
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.100.200
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.111.120
Host is up (0.45s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.99.111.121
Host is up (0.44s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.99.111.122
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.111.13
Host is up (0.46s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.99.111.131
Host is up (0.45s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.99.111.141
Host is up (0.47s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.99.111.15
Host is up (0.45s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.99.111.151
Host is up (0.47s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.99.111.161
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.111.172
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.111.173
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.111.19
Host is up (0.44s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.99.111.2
Host is up (0.45s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.99.111.3
Host is up (0.45s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.99.111.33
Host is up (0.45s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.99.111.7
Host is up (0.44s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.99.112.103
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.112.104
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.112.105
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.112.106
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.112.107
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.112.97
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.112.98
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.113.7
Host is up (0.45s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.99.113.9
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.120.10
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.189.54
Host is up (0.45s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.99.189.56
Host is up (0.43s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.99.189.8
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.189.9
Host is up (0.45s latency).
PORT    STATE SERVICE
445/tcp open  microsoft-ds

Nmap scan report for 10.99.194.134
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.194.18
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.202.100
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.241.20
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.241.21
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.99.241.22
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.26.10
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.26.2
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.26.2
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.26.3
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.26.4
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.26.43
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.26.47
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.26.49
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.26.51
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.26.59
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.26.6
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.26.7
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.34.12
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.34.154
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.34.16
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.34.169
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.34.171
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.34.196
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.34.198
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.34.227
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.34.238
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.39.179
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.39.186
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.39.189
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.39.26
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.39.37
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.39.46
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds

Nmap scan report for 10.100.39.50
Host is up (15s latency).
PORT    STATE  SERVICE
445/tcp closed microsoft-ds
`

				//var oFReader = new FileReader();
				//oFReader.readAsDataURL(document.getElementById("uploadText").files[0]);
				//oFReader.onload = function (oFREvent) {
					//document.getElementById("uploadTextValue").value = oFREvent.target.result; 
					//oFREvent.target.result);
				//var decodedString = atob(oFREvent.target.result.split("base64,")[1]);
					//data:text/html;base64,
					//data:text/plain;base64,
					//document.getElementsByTagName('xmp')[0].innerHTML = decodedString;//oFREvent.target.result;//;
					
					var lines = str.split('\n');
					var fruits = [];
					for(var i = 0;i < lines.length;i++){
					
					if(i%5 == 1) {
					
					var result = lines[i].match(/(\d{1,3}\.){3}\d{1,3}/g);
					var tttttttt =lines[i+3];
					var tempp = tttttttt.indexOf("open");
					if( tempp >= 0)
					document.write(result[0] + "<br>");
					
					}
					
					}
					</script>
逐行读取文件,并使用指定分隔符进行分割

注意123.txt"要为windows格式

<?php
$file = fopen("123.txt", "r") or exit("unable to open file");
while(!feof($file)) {
	$line = fgets($file);
	$hello = explode(',', $line);
	echo $hello[0];
	echo "<br />";
}
fclose($file);
?>
json editor 文本倒序输出

tac命令

[2020-06-13 15:55.17]  ~
[x.x-PC]cat 1
1
2
3
4
5
6
7
                                                                               ✔
────────────────────────────────────────────────────────────────────────────────
[2020-06-13 15:55.18]  ~
[x.x-PC] ➤ tac 1
7
6
5
4
3
2
1
python正则提取出所有url
import re
strings = "http://example.com/path/topage.htmlhttp://twitter.com/p/xyanhshttp://httpget.org/get.zipwww.google.com/privacy.htmlhttps://goodurl.net/"
links = re.findall('http[s]?://(?:(?!http[s]?://)[a-zA-Z]|[0-9]|[$\-_@.&+/]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+', strings)

for link in links:        # Second Example
   print link
python正则提取出所有邮箱地址

将待处理的文本保存为123.txt,执行以下py文件即可

import re
result = {}
if __name__ == "__main__":
    try:
        f = open(r"123.txt", "r")
        content = f.read().lower()
        words = re.findall(
            r'([a-zA-Z0-9_.+-]+@[a-pr-zA-PRZ0-9-]+\.[a-zA-Z0-9-.]+)', content)
       
        for link in words:        # Second Example
            print link
    except BaseException as e:
        print(e)
处理masscan的结果
var str =`[
  {
    "ip": "10.0.133.116",
    "timestamp": "1589793404",
    "ports": [
      {
        "port": 10581,
        "proto": "tcp",
        "status": "open",
        "reason": "syn-ack",
        "ttl": 109
      }
    ]
  },
  {
    "ip": "10.0.133.206",
    "timestamp": "1589793405",
    "ports": [
      {
        "port": 443,
        "proto": "tcp",
        "status": "open",
        "reason": "syn-ack",
        "ttl": 45
      }
    ]
  }
]`
var arr = JSON.parse(str);
document.write("ip,port" + "<br />");
for(var i=0;i<arr.length;i++){
	document.write(arr[i].ip + "," + arr[i].ports[0].port + "<br />");
}
纵向选中文本

其实这个很容易实现,直接把文本内容粘贴到chrome的console处,然后按住alt,使用鼠标选中即可
多种方式文本处理——javascript脚本、bash脚本、python脚本_经验分享_02

AWK

$NF代表最后一个字段

正则

此处参考
teams.txt

Bucks Milwaukee    60 22 0.732
Raptors Toronto    58 24 0.707
76ers Philadelphia 51 31 0.622
Celtics Boston     49 33 0.598
Pacers Indiana     48 34 0.585

使用正则进行匹配

awk '/0.5/ {print $1,$2}' teams.txt

输出包含0.5的行的第一和第二个字段

关系表达式

除了正则外,还有包含语法:

[123.DESKTOP-LS8DC3I]cat teams.txt
Bucks Milwaukee    60 22 0.7532
Raptors Toronto    58 24 0.707
76ers Philadelphia 51 31 0.622
Celtics Boston     49 33 0.598
Pacers Indiana     48 34 0.585
                                                                                                                                          ✔
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[2020-05-17 11:44.13]  ~
[123.DESKTOP-LS8DC3I]awk '$2 ~ /ia/ {print $0}' teams.txt
76ers Philadelphia 51 31 0.622
Pacers Indiana     48 34 0.585

这个命的作用是输出第二个字段中包含ia这个字符串的行

print $0意为打印所有的字段

当然,有包含,那么自然就有不包含,!~

[123.DESKTOP-LS8DC3I]cat teams.txt
Bucks Milwaukee    60 22 0.7532
Raptors Toronto    58 24 0.707
76ers Philadelphia 51 31 0.622
Celtics Boston     49 33 0.598
Pacers Indiana     48 34 0.585
                                                                                                                                          ✔
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[2020-05-17 11:44.36]  ~
[123.DESKTOP-LS8DC3I]awk '$2 ~ /ia/ {print $0}' teams.txt
76ers Philadelphia 51 31 0.622
Pacers Indiana     48 34 0.585

awk支持的关系表达式:

~:包含

!~:不包含

>:大于

<:小于

>=:大于等于

<=:小于等于

=:等于

!=:不等于

范围表达式

显示从RaptorsCeltics范围内的行

[123.DESKTOP-LS8DC3I]cat teams.txt
Bucks Milwaukee    60 22 0.7532
Raptors Toronto    58 24 0.707
76ers Philadelphia 51 31 0.622
Celtics Boston     49 33 0.598
Pacers Indiana     48 34 0.585
                                                                                                                                          ✔
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[2020-05-17 11:34.01]  ~
[123.DESKTOP-LS8DC3I]awk '/Raptors/,/Celtics/ {print $0}' teams.txt
Raptors Toronto    58 24 0.707
76ers Philadelphia 51 31 0.622
Celtics Boston     49 33 0.598

在范围表达式中也可以使用关系表达式:

[123.DESKTOP-LS8DC3I]cat teams.txt
Bucks Milwaukee    60 22 0.7532
Raptors Toronto    58 24 0.707
76ers Philadelphia 51 31 0.622
Celtics Boston     49 33 0.598
Pacers Indiana     48 34 0.585
                                                                                                                                          ✔
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[2020-05-17 11:37.06]  ~
[123.DESKTOP-LS8DC3I]awk '$4 == 31 , $4 == 34 {print $0}' teams.txt
76ers Philadelphia 51 31 0.622
Celtics Boston     49 33 0.598
Pacers Indiana     48 34 0.585

这个语句的意思就是,从第四个字段值为31的行到第四个字段值为34的行都会被打印出来

特殊表达式模式

[123.DESKTOP-LS8DC3I]cat teams.txt
Bucks Milwaukee    60 22 0.7532
Raptors Toronto    58 24 0.707
76ers Philadelphia 51 31 0.622
Celtics Boston     49 33 0.598
Pacers Indiana     48 34 0.585
                                                                                                                                          ✔
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[2020-05-17 11:42.41]  ~
[123.DESKTOP-LS8DC3I]awk 'BEGIN {print "Start Processing."};{print $3};END {print "End Processing."}' teams.txt
Start Processing.
60
58
51
49
48
End Processing.

意思就是告诉系统在开始处理文本前要做什么,处理完毕后要做啥
本例中就是在处理前打印一句话Start Processing.,在处理完毕后打印一句话End Processing.

内置变量

NF:
	该变量用于记录文本中包含几个字段
NR:
	该变量是行的索引
		[123.DESKTOP-LS8DC3I] ➤ cat teams.txt
		Bucks Milwaukee    60 22 0.7532
		Raptors Toronto    58 24 0.707
		76ers Philadelphia 51 31 0.622
		Celtics Boston     49 33 0.598
		Pacers Indiana     48 34 0.585
		                                                                                                                                          ✔
		───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
		[2020-05-17 11:49.11]  ~
		[123.DESKTOP-LS8DC3I] ➤ awk '{print NR}' teams.txt
		1
		2
		3
		4
		5
FILENAME:
	该变量就是处理的文件的名字
FS:
	该变量是用来指定列(字段)分隔符的
RS:
	该变量是用于指定记录(行)分隔符的
OFS:
	该变量存储的是字段分隔符
ORS:
	该变量存储的是记录分隔符

在实战中的作用

它的缺点是处理不了太大的文件

for循环

awk中的for循环中的next有点像高级语言中的continue,当到达next的时候,会忽略下面的语句,回到for语句的判断部分

看下面的例子:

root@kali:~# cat account
张三|000001
李四|000002
root@kali:~# cat cdr
000001|10
000001|20
000002|30
000002|15
root@kali:~# awk -F "|" 'NR==FNR{a[$2]=$0;next}{print a[$1]"|"$2}' account cdr
张三|000001|10
张三|000001|20
李四|000002|30
李四|000002|15

最后一条命令的意思是:

如果当前读取的是第一个文件,那么以第一个文件的第二个字段作为下标,指向第一个文件的当前行,存入数组a中,然后next,上面说明过了,就是continue,不执行接下来的语句,接着处理下一行,这样循环读入,就建立了数组a

000001---->张三|000001
000002---->李四|000002

忘了说一点,就是NRFNR

我直接演示一下他俩的区别:

root@kali:~# cat file1
a b c d
a b c d
root@kali:~# cat file2
aa bb cc dd
aa bb cc dd
root@kali:~# awk '{print NR,$0}' file1
1 a b c d
2 a b c d
root@kali:~# awk '{print FNR,$0}' file2
1 aa bb cc dd
2 aa bb cc dd
root@kali:~# awk '{print NR,$0}' file1 file2
1 a b c d
2 a b c d
3 aa bb cc dd
4 aa bb cc dd
root@kali:~# awk '{print FNR,$0}' file1 file2
1 a b c d
2 a b c d
1 aa bb cc dd
2 aa bb cc dd 

可以看到,在处理单个文件时,NRFNR的输出是没有区别的,但是当我们同时处理两个文件时,就不一样了

对于NR,他把这两个文件联系了起来,输出的行号是1~4

对于FNR,两个文件各有各独立的行号,每个文件都是从第1行开始

因此在awk -F "|" 'NR==FNR{a[$2]=$0;next}{print a[$1]"|"$2}' account cdr中,判断语句NR==FNR的作用就是判断此时处理的是不是第1个文件,如果对这句话不太理解,请看这个:

root@kali:~# awk '{print NR,FNR}' file1 file2
1 1
2 2
3 1
4 2

可以看到,行号相等时表明是第一个文件,由于NRFNR的差异,我们就能根据NR是否等于FNR来判断当前处理的是否为第1个文件

这篇文章可以参考一下:

https://www.cnblogs.com/irockcode/p/7044722.html

写得挺详细的