Apnic介绍
简而言之，Apnic是全球5个地区级的Internet注册机构（RIR）之一，负责亚太地区的以下一些事务：
（1）分配IPv4和IPv6地址空间，AS号
（2）为亚太地区维护Whois数据库
（3）反向DNS指派
（4）在全球范围内作为亚太地区的Internet社区的代表
Apanic IP地址分配信息总表
Apanic提供了每日更新的亚太地区IPv4，IPv6，AS号分配的信息表，访问url是
该文件的格式与具体内容参加
ftp://ftp.apnic.net/pub/apnic/stats/apnic/README.TXT
通过该文件我们能够得到Apnic辖下IPv4地址空间的分配情况。但从该文件中我们只能得到
apnic|TW|asn|1768|2|20020801|allocated
apnic|CN|ipv4|1.2.2.0|256|20110331|assigned
等级机构|获得该IP段的国家/组织|资源类型|起始IP|IP段长度|分配日期|分配状态
Format:
registry|cc|type|start|value|date|status[|extensions...]
Where:
registry The registry from which the data is taken.
For APNIC resources, this will be:
apnic
cc ISO 3166 2-letter code of the organisation to
which the allocation or assignment was made.
May also include the following non-ISO 3166
code:
AP - networks based in more than one
location in the Asia Pacific region
type Type of Internet number resource represented
in this record. One value from the set of
defined strings:
{asn,ipv4,ipv6}
start In the case of records of type 'ipv4' or
'ipv6' this is the IPv4 or IPv6 'first
address' of the range.
value In the case of IPv4 address the count of
hosts for this range. This count does not
have to represent a CIDR range.
但关于省、运营商、地址、路由等信息仍然无法获取
信息总表的扩展
在获取上述总表的基础上，我们通过不断的进行whois，就能得到更多的信息
例如，这对起始IP 1.4.4.0，我们运行whois 1.4.4.0，得到如下的信息
whois 1.4.4.0
[Querying whois.arin.net]
[Redirected to whois.apnic.net]
[Querying whois.apnic.net]
[whois.apnic.net]
% [whois.apnic.net node-1]
% Whois data copyright terms http://www.apnic.net/db/dbcopyright.html
inetnum: 1.4.4.0 – 1.4.4.255
netname: KNET
descr: KNET Techonlogy (BeiJing) Co.,Ltd.
descr: 4,South 4th treet, Zhongguancun,Haidian District,Beijing
country: CN
admin-c: HS1165-AP
tech-c: WL1818-AP
mnt-by: MAINT-CNNIC-AP
mnt-routes: MAINT-CNNIC-AP
mnt-irt: IRT-CNNIC-CN
status: ASSIGNED PORTABLE
changed: hm-changed@apnic.net 20110331
source: APNIC
person: Hanhui Sun
address: 4,South 4th treet, Zhongguancun,Haidian District,Beijing
country: CN
phone: +86-010-58813102
fax-no: +86-010-58812666-126
e-mail: sunhanhui@knet.cn
nic-hdl: HS1165-AP
mnt-by: MAINT-CNNIC-AP
changed: ipas@cnnic.net.cn 20110315
source: APNIC
person: Wenzhe Lu
address: 4,South 4th treet, Zhongguancun,Haidian District,Beijing
country: CN
phone: +86-010-58813017
fax-no: +86-010-58812666-126
e-mail: lwz@knet.cn
nic-hdl: WL1818-AP
mnt-by: MAINT-CNNIC-AP
changed: ipas@cnnic.net.cn 20110315
source: APNIC
其中我们比较关注的信息有：
inetnum：起始和终止IP地址
netname： 网络名（运营商/Org等）
country： 国家
descr： 描述信息（往往含有地域、ISP等信息）
address: 地址信息
通过不断对上述的信息总表的起始IP地址进行whois，我们可以得到详细的IP地址对应的地域、ISP、Org等信息
实现时，在对上述信息总表的起始IP地址进行whois时，需要考虑到两种情况：
（1）对信息总表起始IP地址进行whois时，得到的inetnum中包含的IP地址数目经常会小于信息总表中对应的IP数，因此往往需要多次whois才能得到最终的结果
（2）对信息总表起始IP地址进行whois时，得到的inetnum中的起始IP有可能会小于信息总表中的起始IP，这时也需要进行处理，否则可能使得最终的结果出现IP段交叉的情况
Apnic whois库处理流程总结
为了获取Apnic whois库的完整结果，可以遵循下面的流程：
（1）通过http://ftp.apnic.net/apnic/stats/apnic/delegated-apnic-latest获取Apanic负责分配的所有IPv4地址段的起始地址和长度
（2）通过对上述IP段进行whois操作，得到该IP段的相关信息
（3）whois结果的信息提取、处理和规范化。常见的处理包括国家、省、市、ISP、机构等信息的提取，英文与中文的转化等
通过上述三步，我们就能够得到whois库的结果
#!/bin/bash
#include
#define
TMP=/tmp/apnic_file
FILE=$1
DIR=APNIC
PROG1="whois.sh"
PROG2="merge.sh"
THREAD=30
#function
#main
if [[ -z $1 ]]; then
echo "$(basename $0) "
exit
fi
which whois &>/dev/null
if [[ $? -ne 0 ]]; then
echo "Please install whois(apt-get install whois)"
exit 1
fi
rm -rf $DIR [0-9]* $PROG 2>/dev/null
tail -n +$(awk '/^#!/{if(i){print NR;exit}i++}' $0) "$0" > $PROG1
tail -n +$(awk '/^#!/{if(i==2){print NR;exit}i++}' $0) "$0" > $PROG2
chmod +x $PROG1 $PROG2 2>/dev/null
awk -F"[|]" '/apnic\|CN\|ipv4\|/{print $4,$5}' $FILE > $TMP
awk -vp=$THREAD 'BEGIN{while(getline i}' $TMP
for BLOCK in $(ls [0-9]*); do
./$PROG1 $BLOCK &
done
wait
echo "The whois query is completed"
mkdir -p $DIR/CHINANET $DIR/UNICOM
for dir in $(ls [0-9]*); do
cd $DIR/$dir
for i in $(find . -type f); do
file=${i#*/}
cat $file >> ../$file
done
cd ../..
done
rm -rf $DIR/[0-9]* 2>/dev/null
rm -rf [0-9]* 2>/dev/null
for file in $(find $DIR -type f); do
case $file in
*bug|*error|*print)
echo "ignore $file"
;;
*)
./merge.sh $file
;;
esac
done
rm $PROG1 $PROG2 2>/dev/null
rm /tmp/whois_*
echo "$(basename $0) Completed"
exit 0
#!/bin/bash
#################################################
# 主机数 = 2 ^ (32-mask)
# 所以以2为底取主机数的对数, 就是该mask的值.
#
#MASK=$(cat <<EOF | bc | tail -1
#pow=32;
#define log2(x) {
# if (x<2) return (pow);
# pow--;
# return(log2(x/2));
#}
#log2($CNT)
#EOF
#)
#MASK=$(pow=32;for((i=$CNT;i>1;i=i/2)); do :; ((pow--)); done;echo $pow)
#MASK=$(awk -v c=$CNT 'function log2(x){if(x<2)return(pow);pow--;return(log2(x/2))}BEGIN{pow=32;print log2(c)}')
#################################################
#include
#define
FILE=$1
WHOIS=/tmp/whois_$FILE
DIR=APNIC/$FILE
#function
province(){
case $4 in
FJ*|fj*|FuZhou|fuzhou)
echo "$2/$3" >> $DIR/$1/fujian
;;
GD*)
echo "$2/$3" >> $DIR/$1/guangdong
;;
NM)
echo "$2/$3" >> $DIR/$1/neimenggu
;;
GZ)
echo "$2/$3" >> $DIR/$1/guizhou
;;
NX|NINGXIA)
echo "$2/$3" >> $DIR/$1/ningxia
;;
HL*)
echo "$2/$3" >> $DIR/$1/heilongjiang
;;
SX|TY)
echo "$2/$3" >> $DIR/$1/shanxi
;;
SN|SHAANXI)
echo "$2/$3" >> $DIR/$1/shannxi
;;
HA)
echo "$2/$3" >> $DIR/$1/henan
;;
BJ)
echo "$2/$3" >> $DIR/$1/beijing
;;
CQ)
echo "$2/$3" >> $DIR/$1/chongqing
;;
KM|YN)
echo "$2/$3" >> $DIR/$1/yunan
;;
HB|DIAQOS1)
echo "$2/$3" >> $DIR/$1/hubei
;;
XZ)
echo "$2/$3" >> $DIR/$1/xizang
;;
HE)
echo "$2/$3" >> $DIR/$1/hebei
;;
SD)
echo "$2/$3" >> $DIR/$1/shandong
;;
GS)
echo "$2/$3" >> $DIR/$1/gansu
;;
AH|Anhui)
echo "$2/$3" >> $DIR/$1/anhui
;;
LN)
echo "$2/$3" >> $DIR/$1/liaoning
;;
HN|HUNAN)
echo "$2/$3" >> $DIR/$1/hunan
;;
JS|SZ)
echo "$2/$3" >> $DIR/$1/jiangsu
;;
XJ)
echo "$2/$3" >> $DIR/$1/xinjiang
;;
JX)
echo "$2/$3" >> $DIR/$1/jiangxi
;;
JL)
echo "$2/$3" >> $DIR/$1/jilin
;;
SH|INSURANCE)
echo "$2/$3" >> $DIR/$1/shanghai
;;
GX)
echo "$2/$3" >> $DIR/$1/guangxi
;;
HI)
echo "$2/$3" >> $DIR/$1/hainan
;;
TJ)
echo "$2/$3" >> $DIR/$1/tianjin
;;
SC)
echo "$2/$3" >> $DIR/$1/sichuan
;;
QH|GEERMU)
echo "$2/$3" >> $DIR/$1/qinghai
;;
HK)
echo "$2/$3" >> $DIR/$1/xianggang
;;
ZJ)
echo "$2/$3" >> $DIR/$1/zhejiang
;;
*)
echo "$2/$3" >> $DIR/$1/_other
;;
esac
}
whois_query(){
echo -e "Process[$FILE]\twhois [$1]"
whois $1 > $WHOIS
return $?
}
ntoa(){
awk '{c=256;print int($0/c^3)"."int($0%c^3/c^2)"."int($0%c^3%c^2/c)"."$0%c^3%c^2%c}' <<<$1
}
aton(){
awk '{c=256;split($0,ip,".");print ip[4]+ip[3]*c+ip[2]*c^2+ip[1]*c^3}' <<<$1
}
add_network(){
echo "$2/$3 $1 $4" >> $DIR/print
case $1 in
CHINANET)
province $1 $2 $3 $4
;;
UNICOM)
province $1 $2 $3 $4
;;
CMNET)
echo "$2/$3" >> $DIR/$1
;;
CTTNET)
echo "$2/$3" >> $DIR/$1
;;
CERNET)
echo "$2/$3" >> $DIR/$1
;;
*)
echo "$2/$3 $1 $4" >> $DIR/bug
echo "$2/$3" >> $DIR/others
;;
esac
}
bool_sub(){
START=$HEAD
MASK=32
local NET
local i=$((~0))
while [[ $START -lt $TAIL ]]; do
((i<<=1))
NET=$((HEAD&i))
START=$((~(NET^i)))
((MASK--))
if [[ $START -eq $TAIL ]]; then
return 0
fi
done
return 1
}
do_whois(){
local NET
local i=$((~0))
local j
eval $(awk 'BEGIN{i=256}/^inetnum:/{split($4,ipe,".");ipt=ipe[4]+ipe[3]*i+ipe[2]*i^2+ipe[1]*i^3}END{print "TAIL="ipt}' $WHOIS)
eval $(awk '/^$/{if(i)exit;}\
/^netname:/{i++;split($2,a,"-");isp=a[1];area=a[2];if(isp=="CNC"||isp=="UNI"||isp=="uni")isp="UNICOM";\
if((isp=="UNICOM"&&length(area)) || (isp=="CHINANET"&&length(area)))exit}\
/^mnt-by:.*CNCGROUP/{n=split($2,a,"-");isp="UNICOM";for(x=1;x<=n;x++){if(a[x]=="CNCGROUP"){area=a[x+1];break}};exit}\
/^mnt-by:.*CHINANET/{n=split($2,a,"-");isp="CHINANET";for(x=1;x<=n;x++){if(a[x]=="CHINANET"){area=a[x+1];break}};exit}\
/^mnt-by:.*CERNET/{n=split($2,a,"-");isp="CERNET";for(x=1;x<=n;x++){if(a[x]=="CERNET"){area=a[x+1];break}};exit}\
/^mnt-by: *MAINT-CN-SNXIAN/{isp="CHINANET";area="SN";exit}\
/^netname: *guangzhou-.*-corp/{isp="UNICOM";area="GD";exit}\
/^mnt-lower:.*CERNET/{isp="CERNET";exit}\
/^mnt-lower:.*CHINANET/{n=split($2,a,"-");isp="CHINANET";for(x=1;x<=n;x++){if(a[x]=="CHINANET"){area=a[x+1];break}};exit}\
END{print "ISP="isp";AREA="area}' $WHOIS)
HEAD=$(aton $IP)
bool_sub
if [[ $? -eq 0 ]]; then
add_network $ISP $IP $MASK $AREA
else
j=$((32-MASK))
((i<<=j))
while [[ $NET -ne $HEAD ]]; do
((i>>=1))
NET=$((HEAD&i))
((MASK++))
done
IP=$(ntoa $HEAD)
add_network $ISP $IP $MASK $AREA
TAIL=$((~(NET^i)))
((TAIL++))
IP=$(ntoa $TAIL)
whois_query $IP
if [[ $? -eq 0 ]]; then
do_whois
else
echo "$IP/$MASK" >> $DIR/error
fi
fi
}
#main
FILE=$1
rm -rf $DIR 2>/dev/null
mkdir -p $DIR/CHINANET $DIR/UNICOM
while read IP CNT; do
START=$(aton $IP)
END=$((START+CNT-1))
TAIL=0
MASK_MAX=$(pow=32;for((i=$CNT;i>1;i>>=1)); do :; ((pow--)); done;echo $pow)
while [[ $TAIL -lt $END ]]; do
whois_query $IP
if [[ $? -eq 0 ]]; then
do_whois
((TAIL++))
IP=$(ntoa $TAIL)
else
echo "$IP/$MASK" >> $DIR/error
fi
done
done < $FILE
exit 0
#!/bin/bash
if [[ -z $1 ]]; then
echo "file not found"
exit
fi
if [[ ! -f $1 ]]; then
echo "$1 is not exsit"
exit
fi
TMP=/tmp/merge
while :; do
awk -F"/" '
function ntoa(n){c=256;return int(n/c^3)"."int(n%c^3/c^2)"."int(n%c^3%c^2/c)"."n%c^3%c^2%c}
function aton(d){c=256;split(d,ip,".");return ip[4]+ip[3]*c+ip[2]*c^2+ip[1]*c^3}
function ntobc(a,b){e=compl(0);f=lshift(e,32-b);s=and(a,f);return compl(xor(s,f))}
function ntosub(j,k){g=compl(0);h=lshift(g,32-k);return and(j,h)}
NR>1{
if($1==ntoa(bc+1) && $2==mask && ip_int==ntosub(ip_int,$2-1)){
mask=$2-1;bc=ntobc(ip_int,mask);
next;
}else{
print add"/"mask
}
}
{add=$1;ip_int=aton($1);mask=$2;bc=ntobc(ip_int,mask)}
END{print add"/"mask}
' $1 > $TMP
FILE_SIZE=$(ls -l $1 2>/dev/null | awk '{print $5}')
TMP_SIZE=$(ls -l $TMP 2>/dev/null | awk '{print $5}')
if [[ $FILE_SIZE -eq $TMP_SIZE ]]; then
break
fi
cp $TMP $1
done
exit 0
==============================
#!/bin/bash
# download from apnic
rm -f delegated-apnic-latest
wget http://ftp.apnic.net/apnic/stats/apnic/delegated-apnic-latest
# IPs allocated to china.
grep 'apnic|CN|ipv4|' delegated-apnic-latest | cut -f 4 -d'|' > delegated-apnic-CN
# get detail of echo IP from apnic database.
rm -f apnic_CN.txt
while read ip
do
# query apnic database
echo "query who is $ip"
whois -h whois.apnic.net $ip > tmp.txt
grep inetnum tmp.txt >> apnic_CN.txt # IP range
grep netname tmp.txt >> apnic_CN.txt # netname which include sp information
grep descr tmp.txt >> apnic_CN.txt # description which include province information
echo "" >> apnic_CN.txt
done < delegated-apnic-CN
# clean up
rm -f tmp.txt
rm -f delegated-apnic-latest
rm -f delegated-apnic-CN
APNIC是管理亚太地区IP地址分配的机构，它有着丰富准确的IP地址分配库，同时这些信息也是对外公开的！
一、在Linux下获得一些电信运营商的IP地址段的情况：
1、编译安装软件
shell> wget http://ftp.apnic.net/apnic/dbase/tools/ripe-dbase-client-v3.tar.gz
shell> tar xzvf ripe-dbase-client-v3.tar.gz
shell> cd whois-3.1
shell> ./configure
shell> make
2、开始获取IP地址段
中国网通：
shell> ./whois3 -h whois.apnic.net -l -i mb MAINT-CNCGROUP > /root/cncrange.txt
中国电信：
shell> ./whois3 -h whois.apnic.net -l -i mb MAINT-CHINANET > /root/chinanetrange.txt
中国铁通：
shell> ./whois3 -h whois.apnic.net -l -i mb MAINT-CN-CRTC > /root/crtcrange.txt
专门获取中国网通路由表
shell> ./whois3 -h whois.apnic.net -l -i mb MAINT-CNCGROUP-RR > /root/cncrr.txt
注：MAINT-CNCGROUP 为中国网通在apnic的自治系统号
如果想得到具体的服务商比如山东网通的 就用MAINT-CNCGROUP-SD
具体查看http://www.apnic.net/index.html
其他查询选项 请查看 http://www.apnic.net/apnic-info/whois_search
3、过滤提取IP段
以中国网通为例:
shell> cat cncip.txt | grep “inetnum” | sed ‘s/inetnum: //g’
中国网通路由表
shell> cat /root/cnciprt.txt | grep route | sed ‘s/route://g’ | sed ‘s/. //g’
用下面的工具进行整理更方便LFHIPConverter subnettools102
如果做智能dns的话 用下面的方法：
获取中国CNC网通地址列表字段
./whois3 -h whois.apnic.net -l -i mb MAINT-CNCGROUP | grep “descr” | grep “Reverse” | awk -F “for” ‘{if ($2!=”") print $2}’| sort -n | awk ‘BEGIN{print “acl /”CNC/” ‘{‘”}{print $1″;”}END{print “‘}’;”}’ > /var/named/cnc_acl.conf获取中国电信CTC 地址列表字段
./whois3 -h whois.apnic.net -l -i mb MAINT-CHINANET | grep “descr” | grep “Reverse” | awk -F “for” ‘{if ($2!=”") print $2}’| sort -n | awk ‘BEGIN{print “acl /”CTC/” ‘{‘”}{print $1″;”}END{print “‘}’;”}’ > /var/named/ctc_acl.conf
二、使用ripe-whois3获得电信，网通等运营商的ip地址（以下方法没有测试，摘抄于网络）
Linux中下载安装
http://ftp.apnic.net/apnic/dbase/tools/ripe-whois-tools-2.3.tar.gz
tar xzvf ripe-whois-tools-2.3.tar.gz
cd /usr/ports/net/ripe-whois3
make install clean
rehash
查询:
中国网通: whois3 -h whois.apnic.net -l -i mb MAINT-CNCGROUP
中国电信: whois3 -h whois.apnic.net -l -i mb MAINT-CHINANET
中国铁通: whois3 -h whois.apnic.net -l -i mb MAINT-CNC-CRTC
三、自动获取任意国家地区ip段（需要自己根据实际情况修改）
#!/bin/sh
FILE=./ip_apnic
rm -f $FILE
wget http://ftp.apnic.net/apnic/stats/apnic/delegated-apnic-latest -O $FILE
grep ‘apnic|CN|ipv4|’ $FILE | cut -f 4,5 -d’|'|sed -e ‘s/|/ /g’ | while read ip cnt
do
echo $ip:$cnt
mask=$(cat << EOF | bc | tail -1
pow=32;
define log2(x) {
if (x<=1) return (pow);
pow–;
return(log2(x/2));
}
log2($cnt)
EOF)
echo $ip/$mask>> cn.net
NETNAME=`whois $ip@whois.apnic.net | sed -e ‘/./{H;$!d;}’ -e ‘x;/netnum/!d’ |grep ^netname | sed -e ‘s/.*: /(.*/)//1/g’ | sed -e ‘s/-.*//g’`
case $NETNAME in
CHINANET|CNCGROUP)
echo $ip/$mask >> $NETNAME
;;
#如果你还要其他 ISP , 请在这边加上去即可,透过 apnic whois , 你可以知道他的 NETNAME OTHER_NETNAME_here)
;;
esac
done
================== End
