自从过完年之后没有发过什么文章了，一直在写一个快捷酒店房间采集程序，现在已经做完了七天连锁的和如家的房间信息采集，现在会陆续将代码贴上来
我的python工程的目录如下图所示：
程序是使用python2.7+mongod+threadpool完成的，所以你还需要一个mongodb数据库和下载一个threadpool线程池库
一，我们首先在resources目录下放入两个个xml文件，第一个文件为homeinns.xml，这是初始化如家酒店的相关城市的信息
<?xml version="1.0"?>
<td>
<dd title="0100" class="hover" style="display: block;">beijing<span>北京</span></dd><dd title="0210" class="" style="display: block;">shanghai<span>上海</span></dd><dd title="0571" style="display: block;">hangzhou<span>杭州</span></dd><dd title="0290" style="display: block;">xian<span>西安</span></dd><dd title="0220" style="display: block;">tianjin<span>天津</span></dd><dd title="0532" style="display: block;">qingdao<span>青岛</span></dd><dd title="0240" style="display: block;">shenyang<span>沈阳</span></dd><dd title="0270" style="display: block;">wuhan<span>武汉</span></dd><dd title="0200" style="display: block;">guangzhou<span>广州</span></dd><dd title="0250" style="display: block;">nanjing<span>南京</span></dd><dd title="0531" style="display: none;">jinan<span>济南</span></dd><dd title="0311" style="display: none;">shijiazhuang<span>石家庄</span></dd><dd title="0411" style="display: none;">dalian<span>大连</span></dd><dd title="0592" style="display: none;">xiamen<span>厦门</span></dd><dd title="0280" style="display: none;">chengdu<span>成都</span></dd><dd title="0512A" style="display: none;">suzhou<span>苏州</span></dd><dd title="0451" style="display: none;">haerbin<span>哈尔滨</span></dd><dd title="0755" style="display: none;">shenzhen<span>深圳</span></dd><dd title="0991" style="display: none;">wulumuqi<span>乌鲁木齐</span></dd><dd title="0510A" style="display: none;">wuxi<span>无锡</span></dd><dd title="0574" style="display: none;">ningbo<span>宁波</span></dd><dd title="0351" style="display: none;">taiyuan<span>太原</span></dd><dd title="0931" style="display: none;">lanzhou<span>兰州</span></dd><dd title="0230" style="display: none;">chongqing<span>重庆</span></dd><dd title="0551" style="display: none;">hefei<span>合肥</span></dd><dd title="0371" style="display: none;">zhengzhou<span>郑州</span></dd><dd title="0791" style="display: none;">nanchang<span>南昌</span></dd><dd title="0431" style="display: none;">changchun<span>长春</span></dd><dd title="0519" style="display: none;">changzhou<span>常州</span></dd><dd title="0471" style="display: none;">huhehaote<span>呼和浩特</span></dd><dd title="0898A" style="display: none;">sanya<span>三亚</span></dd><dd title="0871" style="display: none;">kunming<span>昆明</span></dd><dd title="0518" style="display: none;">lianyungang<span>连云港</span></dd><dd title="0513A" style="display: none;">nantong<span>南通</span></dd><dd title="0516" style="display: none;">xuzhou<span>徐州</span></dd><dd title="0591" style="display: none;">fuzhou<span>福州</span></dd><dd title="0539" style="display: none;">linyi<span>临沂</span></dd><dd title="0514" style="display: none;">yangzhou<span>扬州</span></dd><dd title="0535A" style="display: none;">yantai<span>烟台</span></dd><dd title="0538" style="display: none;">taian<span>泰安</span></dd><dd title="0731" style="display: none;">changsha<span>长沙</span></dd><dd title="0631A" style="display: none;">weihai<span>威海</span></dd><dd title="0851" style="display: none;">guiyang<span>贵阳</span></dd><dd title="0898" style="display: none;">haikou<span>海口</span></dd><dd title="0523" style="display: none;">taizhou<span>泰州</span></dd><dd title="0511" style="display: none;">zhenjiang<span>镇江</span></dd><dd title="0754" style="display: none;">shantou<span>汕头</span></dd><dd title="0757" style="display: none;">foshan<span>佛山</span></dd><dd title="0527" style="display: none;">suqian<span>宿迁</span></dd><dd title="0310" style="display: none;">handan<span>邯郸</span></dd><dd title="0316" style="display: none;">langfang<span>廊坊</span></dd><dd title="0771" style="display: none;">nanning<span>南宁</span></dd><dd title="0765" style="display: none;">shunde<span>顺德</span></dd><dd title="0517" style="display: none;">huaian<span>淮安</span></dd><dd title="0553" style="display: none;">wuhu<span>芜湖</span></dd><dd title="0633" style="display: none;">rizhao<span>日照</span></dd><dd title="0536" style="display: none;">weifang<span>潍坊</span></dd><dd title="0533" style="display: none;">zibo<span>淄博</span></dd><dd title="0315" style="display: none;">tangshan<span>唐山</span></dd><dd title="0312" style="display: none;">baoding<span>保定</span></dd><dd title="0432" style="display: none;">jilin<span>吉林</span></dd><dd title="0971" style="display: none;">xining<span>西宁</span></dd><dd title="0512C" style="display: none;">kunshan<span>昆山</span></dd><dd title="0510B" style="display: none;">jiangyin<span>江阴</span></dd><dd title="0635" style="display: none;">liaocheng<span>聊城</span></dd><dd title="0534" style="display: none;">dezhou<span>德州</span></dd><dd title="0530" style="display: none;">heze<span>菏泽</span></dd><dd title="0472" style="display: none;">baotou<span>包头</span></dd><dd title="0452" style="display: none;">qiqihaer<span>齐齐哈尔</span></dd><dd title="0773" style="display: none;">guilin<span>桂林</span></dd><dd title="0951" style="display: none;">yinchuan<span>银川</span></dd><dd title="0752" style="display: none;">huizhou<span>惠州</span></dd><dd title="0756" style="display: none;">zhuhai<span>珠海</span></dd><dd title="0769" style="display: none;">dongguan<span>东莞</span></dd><dd title="0515" style="display: none;">yancheng<span>盐城</span></dd><dd title="0512B" style="display: none;">changshu<span>常熟</span></dd><dd title="0574B" style="display: none;">cixiningbo<span>慈溪(宁波)</span></dd><dd title="0575" style="display: none;">shaoxing<span>绍兴</span></dd><dd title="0537" style="display: none;">zoucheng<span>邹城</span></dd><dd title="0537A" style="display: none;">jining<span>济宁</span></dd><dd title="0546" style="display: none;">dongying<span>东营</span></dd><dd title="0543" style="display: none;">binzhou<span>滨州</span></dd><dd title="0719" style="display: none;">shiyan<span>十堰</span></dd><dd title="0717" style="display: none;">yichang<span>宜昌</span></dd><dd title="0335" style="display: none;">qinhuangdao<span>秦皇岛</span></dd><dd title="0352" style="display: none;">datong<span>大同</span></dd><dd title="0355" style="display: none;">changzhi<span>长治</span></dd><dd title="0359" style="display: none;">yuncheng<span>运城</span></dd><dd title="0415" style="display: none;">dandong<span>丹东</span></dd><dd title="0760" style="display: none;">zhongshan<span>中山</span></dd><dd title="0519A" style="display: none;">liyang<span>溧阳</span></dd><dd title="0520" style="display: none;">zhangjiagang<span>张家港</span></dd><dd title="0510C" style="display: none;">yixing<span>宜兴</span></dd><dd title="0573" style="display: none;">jiaxing<span>嘉兴</span></dd><dd title="0552" style="display: none;">bengbu<span>蚌埠</span></dd><dd title="0596" style="display: none;">zhangzhou<span>漳州</span></dd><dd title="0792" style="display: none;">jiujiang<span>九江</span></dd><dd title="0798" style="display: none;">jingdezheng<span>景德镇</span></dd><dd title="0632" style="display: none;">zaozhuang<span>枣庄</span></dd><dd title="0378" style="display: none;">kaifeng<span>开封</span></dd><dd title="0379" style="display: none;">luoyang<span>洛阳</span></dd><dd title="0317" style="display: none;">cangzhou<span>沧州</span></dd><dd title="0413" style="display: none;">fushun<span>抚顺</span></dd><dd title="0772" style="display: none;">liuzhou<span>柳州</span></dd><dd title="0290A" style="display: none;">xianyang<span>咸阳</span></dd><dd title="0917" style="display: none;">baoji<span>宝鸡</span></dd><dd title="0933" style="display: none;">pingliang<span>平凉</span></dd><dd title="0937A" style="display: none;">jiuquan<span>酒泉</span></dd><dd title="0663A" style="display: none;">puning<span>普宁</span></dd><dd title="0668" style="display: none;">maomin<span>茂名</span></dd><dd title="0751" style="display: none;">shaoguan<span>韶关</span></dd><dd title="0758" style="display: none;">zhaoqing<span>肇庆</span></dd><dd title="0513C" style="display: none;">haian<span>海安</span></dd><dd title="0513D" style="display: none;">rugaonantong<span>如皋(南通)</span></dd><dd title="0523A" style="display: none;">jingjiang<span>靖江</span></dd><dd title="0513B" style="display: none;">haimeng<span>海门</span></dd><dd title="0512D" style="display: none;">taicang<span>太仓</span></dd><dd title="0572" style="display: none;">huzhou<span>湖州</span></dd><dd title="0573B" style="display: none;">haining<span>海宁</span></dd><dd title="0574A" style="display: none;">yuyao<span>余姚</span></dd><dd title="0575A" style="display: none;">zhuji<span>诸暨</span></dd><dd title="0579A" style="display: none;">yiwu<span>义乌</span></dd><dd title="0579" style="display: none;">jinhua<span>金华</span></dd><dd title="0550" style="display: none;">chuzhou<span>滁州</span></dd><dd title="0554" style="display: none;">huainan<span>淮南</span></dd><dd title="0561" style="display: none;">huaibei<span>淮北</span></dd><dd title="0562" style="display: none;">tongling<span>铜陵</span></dd><dd title="0556" style="display: none;">anqing<span>安庆</span></dd><dd title="0559" style="display: none;">huangshan<span>黄山</span></dd><dd title="0555" style="display: none;">maanshan<span>马鞍山</span></dd><dd title="0595A" style="display: none;">quanzhou<span>泉州</span></dd><dd title="0533E" style="display: none;">linzizibo<span>临淄(淄博)</span></dd><dd title="0535C" style="display: none;">zhaoyuanyantai<span>招远(烟台)</span></dd><dd title="0535D" style="display: none;">longkouyantai<span>龙口(烟台)</span></dd><dd title="0535E" style="display: none;">haiyangyantai<span>海阳(烟台)</span></dd><dd title="0536B" style="display: none;">shouguangweifang<span>寿光(潍坊)</span></dd><dd title="0537B" style="display: none;">qufu<span>曲阜</span></dd><dd title="0535" style="display: none;">penglaiyantai<span>蓬莱(烟台)</span></dd><dd title="0632A" style="display: none;">tengzhou<span>滕州</span></dd><dd title="0634" style="display: none;">laiwu<span>莱芜</span></dd><dd title="0535B" style="display: none;">laiyangyantai<span>莱阳(烟台)</span></dd><dd title="0374" style="display: none;">xuchang<span>许昌</span></dd><dd title="0377" style="display: none;">nanyang<span>南阳</span></dd><dd title="0373" style="display: none;">xinxiang<span>新乡</span></dd><dd title="0710" style="display: none;">xiangyang<span>襄阳</span></dd><dd title="0734" style="display: none;">hengyang<span>衡阳</span></dd><dd title="0313" style="display: none;">zhangjiakou<span>张家口</span></dd><dd title="0318" style="display: none;">hengshui<span>衡水</span></dd><dd title="0319" style="display: none;">xingtai<span>邢台</span></dd><dd title="0353" style="display: none;">yangquan<span>阳泉</span></dd><dd title="0354" style="display: none;">jinzhong<span>晋中</span></dd><dd title="0356" style="display: none;">jincheng<span>晋城</span></dd><dd title="0357" style="display: none;">linfen<span>临汾</span></dd><dd title="0350" style="display: none;">xinzhou<span>忻州</span></dd><dd title="0476" style="display: none;">chifeng<span>赤峰</span></dd><dd title="0417B" style="display: none;">bayuquanyingkou<span>鲅鱼圈(营口)</span></dd><dd title="0419" style="display: none;">liaoyang<span>辽阳</span></dd><dd title="0421" style="display: none;">chaoyang<span>朝阳</span></dd><dd title="0427" style="display: none;">panjin<span>盘锦</span></dd><dd title="0429" style="display: none;">huludao<span>葫芦岛</span></dd><dd title="0416" style="display: none;">jinzhou<span>锦州</span></dd><dd title="0412" style="display: none;">anshan<span>鞍山</span></dd><dd title="0439" style="display: none;">baishan<span>白山</span></dd><dd title="0453" style="display: none;">mudanjiang<span>牡丹江</span></dd><dd title="0454" style="display: none;">jiamusi<span>佳木斯</span></dd><dd title="0455" style="display: none;">suihua<span>绥化</span></dd><dd title="0459" style="display: none;">daqing<span>大庆</span></dd><dd title="0779" style="display: none;">beihai<span>北海</span></dd><dd title="0280A" style="display: none;">dujiangyan(chengdu)<span>都江堰(成都)</span></dd><dd title="0830" style="display: none;">luzhou<span>泸州</span></dd><dd title="0833" style="display: none;">leshan<span>乐山</span></dd><dd title="0852" style="display: none;">zunyi<span>遵义</span></dd><dd title="0691" style="display: none;">xishuangbanna<span>西双版纳</span></dd><dd title="0888" style="display: none;">lijiang<span>丽江</span></dd><dd title="0891" style="display: none;">lasa<span>拉萨</span></dd><dd title="0912A" style="display: none;">jingbianyulin<span>靖边(榆林)</span></dd><dd title="0916" style="display: none;">hanzhong<span>汉中</span></dd><dd title="0936" style="display: none;">zhangye<span>张掖</span></dd><dd title="0937" style="display: none;">jiayuguan<span>嘉峪关</span></dd><dd title="0935" style="display: none;">wuwei<span>武威</span></dd><dd title="0994" style="display: none;">changji<span>昌吉</span></dd><dd title="0230A" style="display: none;">wanzhouchongqing<span>万州(重庆)</span></dd><dd title="0663" style="display: none;">jieyang<span>揭阳</span></dd><dd title="0750" style="display: none;">jiangmen<span>江门</span></dd><dd title="0753" style="display: none;">meizhou<span>梅州</span></dd><dd title="0759" style="display: none;">zhanjiang<span>湛江</span></dd><dd title="0762" style="display: none;">heyuan<span>河源</span></dd><dd title="0766" style="display: none;">yunfu<span>云浮</span></dd><dd title="0768" style="display: none;">chaozhou<span>潮州</span></dd><dd title="0511A" style="display: none;">jurong(zhenjiang)<span>句容(镇江)</span></dd><dd title="0513F" style="display: none;">rudong<span>如东</span></dd><dd title="0514B" style="display: none;">yizheng<span>仪征</span></dd><dd title="0515A" style="display: none;">dongtai<span>东台</span></dd><dd title="0515B" style="display: none;">jianhu<span>建湖</span></dd><dd title="0515C" style="display: none;">sheyang<span>射阳</span></dd><dd title="0515D" style="display: none;">dafeng(yancheng)<span>大丰(盐城)</span></dd><dd title="0516A" style="display: none;">pizhou<span>邳州</span></dd><dd title="0519B" style="display: none;">jintan<span>金坛</span></dd><dd title="0513E" style="display: none;">qidong<span>启东</span></dd><dd title="0571A" style="display: none;">linan<span>临安</span></dd><dd title="0571B" style="display: none;">tongluhangzhou<span>桐庐(杭州)</span></dd><dd title="0571C" style="display: none;">fuyanghangzhou<span>富阳(杭州)</span></dd><dd title="0576A" style="display: none;">tiantai(taizhou)<span>天台(台州)</span></dd><dd title="0580" style="display: none;">zhoushan<span>舟山</span></dd><dd title="0557" style="display: none;">suzhou<span>宿州</span></dd><dd title="0558" style="display: none;">bozhou<span>亳州</span></dd><dd title="0593" style="display: none;">ningde<span>宁德</span></dd><dd title="0594" style="display: none;">putian<span>莆田</span></dd><dd title="0597" style="display: none;">longyan<span>龙岩</span></dd><dd title="0599" style="display: none;">wuyishan<span>武夷山</span></dd><dd title="0595B" style="display: none;">shishi<span>石狮</span></dd><dd title="0793" style="display: none;">shangrao<span>上饶</span></dd><dd title="0794" style="display: none;">fuzhou<span>抚州</span></dd><dd title="0797" style="display: none;">ganzhou<span>赣州</span></dd><dd title="0790" style="display: none;">xinyu<span>新余</span></dd><dd title="0799" style="display: none;">pingxiang<span>萍乡</span></dd><dd title="0531A" style="display: none;">zhangqiujinan<span>章丘(济南)</span></dd><dd title="0533A" style="display: none;">yiyuanzibo<span>沂源(淄博)</span></dd><dd title="0533B" style="display: none;">boshanzibo<span>博山(淄博)</span></dd><dd title="0533C" style="display: none;">zhoucunzibo<span>周村(淄博)</span></dd><dd title="0533D" style="display: none;">zichuanzibo<span>淄川(淄博)</span></dd><dd title="0535F" style="display: none;">mupingyantai<span>牟平(烟台)</span></dd><dd title="0536A" style="display: none;">zhuchengweifang<span>诸城(潍坊)</span></dd><dd title="0538A" style="display: none;">feichengtaian<span>肥城(泰安)</span></dd><dd title="0539A" style="display: none;">yishuilinyi<span>沂水(临沂)</span></dd><dd title="0539B" style="display: none;">cangshanlinyi<span>苍山(临沂)</span></dd><dd title="0543A" style="display: none;">zoupingbinzhou<span>邹平(滨州)</span></dd><dd title="0631" style="display: none;">rushanweihai<span>乳山(威海)</span></dd><dd title="0631C" style="display: none;">wendengweihai<span>文登(威海)</span></dd><dd title="0631D" style="display: none;">shidaoweihairongcheng<span>石岛(威海荣成)</span></dd><dd title="0635A" style="display: none;">linqingliaocheng<span>临清(聊城)</span></dd><dd title="0635B" style="display: none;">yangguliaocheng<span>阳谷(聊城)</span></dd><dd title="0631B" style="display: none;">rongchengweihai<span>荣成(威海)</span></dd><dd title="0370" style="display: none;">shangqiu<span>商丘</span></dd><dd title="0372" style="display: none;">anyang<span>安阳</span></dd><dd title="0391" style="display: none;">jiaozuo<span>焦作</span></dd><dd title="0394" style="display: none;">zhoukou<span>周口</span></dd><dd title="0395" style="display: none;">luohe<span>漯河</span></dd><dd title="0393" style="display: none;">puyang<span>濮阳</span></dd><dd title="0716" style="display: none;">jingzhou<span>荆州</span></dd><dd title="0718" style="display: none;">enshi<span>恩施</span></dd><dd title="0731A" style="display: none;">zhuzhou<span>株洲</span></dd><dd title="0731B" style="display: none;">xiangtan<span>湘潭</span></dd><dd title="0737" style="display: none;">yiyang<span>益阳</span></dd><dd title="0744" style="display: none;">zhangjiajie<span>张家界</span></dd><dd title="0730" style="display: none;">yueyang<span>岳阳</span></dd><dd title="0311B" style="display: none;">xinjishijiazhuang<span>辛集(石家庄)</span></dd><dd title="0314" style="display: none;">chengde<span>承德</span></dd><dd title="0315A" style="display: none;">qianan<span>迁安</span></dd><dd title="0349" style="display: none;">shuozhou<span>朔州</span></dd><dd title="0475" style="display: none;">tongliao<span>通辽</span></dd><dd title="0478" style="display: none;">bayannaoer<span>巴彦淖尔</span></dd><dd title="0479" style="display: none;">xilinhaote<span>锡林浩特</span></dd><dd title="0480" style="display: none;">eerduosi<span>鄂尔多斯</span></dd><dd title="0240A" style="display: none;">tieling<span>铁岭</span></dd><dd title="0414" style="display: none;">benxi<span>本溪</span></dd><dd title="0417A" style="display: none;">dashiqiaoyingkou<span>大石桥(营口)</span></dd><dd title="0418" style="display: none;">fuxin<span>阜新</span></dd><dd title="0417" style="display: none;">yingkou<span>营口</span></dd><dd title="0434" style="display: none;">siping<span>四平</span></dd><dd title="0436" style="display: none;">baicheng<span>白城</span></dd><dd title="0438" style="display: none;">songyuan<span>松原</span></dd><dd title="0435" style="display: none;">tonghua<span>通化</span></dd><dd title="0433" style="display: none;">yanji<span>延吉</span></dd><dd title="0453B" style="display: none;">suifenhemudanjiang<span>绥芬河(牡丹江)</span></dd><dd title="0458" style="display: none;">yichun<span>伊春</span></dd><dd title="0468" style="display: none;">hegang<span>鹤岗</span></dd><dd title="0770" style="display: none;">fangchenggang<span>防城港</span></dd><dd title="0775" style="display: none;">yulin<span>玉林</span></dd><dd title="0775A" style="display: none;">guiping<span>桂平</span></dd><dd title="0280B" style="display: none;">meishan(chengdu)<span>眉山（成都）</span></dd><dd title="0280C" style="display: none;">mianzhu<span>绵竹</span></dd><dd title="0812" style="display: none;">panzhihua<span>攀枝花</span></dd><dd title="0816" style="display: none;">mianyang<span>绵阳</span></dd><dd title="0834" style="display: none;">xichang<span>西昌</span></dd><dd title="0838" style="display: none;">deyang<span>德阳</span></dd><dd title="0692" style="display: none;">ruili<span>瑞丽</span></dd><dd title="0854" style="display: none;">qujing<span>曲靖</span></dd><dd title="0870" style="display: none;">zhaotong<span>昭通</span></dd><dd title="0873" style="display: none;">kaiyuan<span>开远</span></dd><dd title="0875" style="display: none;">tengchongbaoshan<span>腾冲(保山)</span></dd><dd title="0877" style="display: none;">yuxi<span>玉溪</span></dd><dd title="0879" style="display: none;">puer<span>普洱</span></dd><dd title="0910" style="display: none;">yanan<span>延安</span></dd><dd title="0912" style="display: none;">yulin<span>榆林</span></dd><dd title="0913" style="display: none;">weinan<span>渭南</span></dd><dd title="0919" style="display: none;">tongchuan<span>铜川</span></dd><dd title="0932" style="display: none;">dingxi<span>定西</span></dd><dd title="0934" style="display: none;">qingyang<span>庆阳</span></dd><dd title="0935A" style="display: none;">jinchang<span>金昌</span></dd><dd title="0937B" style="display: none;">dunhuangjiayuguan<span>敦煌(嘉峪关)</span></dd><dd title="0938" style="display: none;">tianshui<span>天水</span></dd><dd title="0953A" style="display: none;">zhongwei<span>中卫</span></dd><dd title="0954" style="display: none;">guyuan<span>固原</span></dd><dd title="0953" style="display: none;">wuzhong<span>吴忠</span></dd><dd title="0991A" style="display: none;">tulufan<span>吐鲁番</span></dd><dd title="0993" style="display: none;">shihezi<span>石河子</span></dd><dd title="0912B" style="display: none;">fukang<span>阜康</span></dd><dd title="0999" style="display: none;">yining<span>伊宁</span></dd><dd title="0998" style="display: none;">kashi<span>喀什</span></dd><dd title="0898B" style="display: none;">lingshui<span>陵水</span></dd><dd title="0898C" style="display: none;">qionghai<span>琼海</span></dd></td>
第二个文件为mongodb-config.xml，内容如下:<mongos>
<mainMongo>
<host>localhost</host>
<port>27017</port>
<connectTimeout>3000</connectTimeout>
<connectionsPerHost>30</connectionsPerHost>
<dbName>jhnet</dbName>
</mainMongo>
</mongos>
加入文件完毕之后，我们需要在init包中加入文件province_city.xml，内容如下(这可是哥手动打上去的啊):
<provinces>
<province name="北京市" province_pinyinname="beijing" alias="直辖市">
<citys>
<city name="北京" city_pinyinname="beijing" alias="直辖市">
</city>
</citys>
</province>
<province name="天津市" province_pinyinname="tianjin" alias="直辖市">
<citys>
<city name="天津" city_pinyinname="tianjin" alias="直辖市">
</city>
</citys>
</province>
<province name="上海市" province_pinyinname="shanghai" alias="直辖市">
<citys>
<city name="上海" city_pinyinname="shanghai" alias="直辖市">
</city>
</citys>
</province>
<province name="重庆市" province_pinyinname="chongqing" alias="直辖市">
<citys>
<city name="重庆" city_pinyinname="chongqing" alias="直辖市">
</city>
</citys>
</province>
<province name="安徽省" province_pinyinname="anhui" >
<citys>
<city name="合肥" city_pinyinname="hefei" alias="省会" />
<city name="宿州" city_pinyinname="suzhou" alias=""/>
<city name="淮北" city_pinyinname="huaibei" alias=""/>
<city name="阜阳" city_pinyinname="fuyang" />
<city name="蚌埠" city_pinyinname="bengbu"/>
<city name="淮南" city_pinyinname="huainan" />
<city name="滁州" city_pinyinname="chuzhou"/>
<city name="马鞍山" city_pinyinname="maanshang"/>
<city name="芜湖" city_pinyinname="wuhu"/>
<city name="铜陵" city_pinyinname="tongling" />
<city name="安庆" city_pinyinname="anqing" />
<city name="黄山" city_pinyinname="huangshan"/>
<city name="六安" city_pinyinname="liuan"/>
<city name="池州" city_pinyinname="chizhou"/>
<city name="宣城" city_pinyinname="xuancheng"/>
<city name="毫州" city_pinyinname="haozhou"/>
</citys>
</province>
<province name="福建省" province_pinyinname="fujian" >
<citys>
<city name="厦门" city_pinyinname="xiamen" />
<city name="福州" city_pinyinname = "fuzhou"/>
<city name="南平" city_pinyinname="nanping"/>
<city name="三明" city_pinyinname="sanming"/>
<city name="莆田" city_pinyinname="putian"/>
<city name="泉州" city_pinyinname="quanzhou"/>
<city name="漳州" city_pinyinname="zhangzhou"/>
<city name="龙岩" city_pinyinname="longyan"/>
<city name="宁德" city_pinyinname="ningde"/>
</citys>
</province>
<province name="甘肃省" province_pinyinname="gansu">
<citys>
<city name="兰州" city_pinyinname="lanzhou"/>
<city name="嘉峪关" city_pinyinname="jiayuguan"/>
<city name="金昌" city_pinyinname="jinchang"/>
<city name="白银" city_pinyinname="baiyin"/>
<city name="天水" city_pinyinname="tianshui"/>
<city name="酒泉" city_pinyinname="jiuquan"/>
<city name="张掖" city_pinyinname="zhangye"/>
<city name="武威" city_pinyinname="wuwei"/>
<city name="庆阳" city_pinyinname="qingyang"/>
<city name="平凉" city_pinyinname="pingliang"/>
<city name="定西" city_pinyinname="dingxi"/>
<city name="陇南" city_pinyinname="longnan"/>
</citys>
</province>
<province name="广东省" province_pinyinname="guangdong">
<citys>
<city name="广州" city_pinyinname="guangzhou"/>
<city name="深圳" city_pinyinname="shenzhen"/>
<city name="清远" city_pinyinname="qingyuan"/>
<city name="韶关" city_pinyinname="shaoguan"/>
<city name="河源" city_pinyinname="heyuan"/>
<city name="梅州" city_pinyinname="meizhou"/>
<city name="潮州" city_pinyinname="chaozhou"/>
<city name="汕头" city_pinyinname="shantou"/>
<city name="揭阳" city_pinyinname="jieyang"/>
<city name="汕尾" city_pinyinname="shanwei"/>
<city name="惠州" city_pinyinname="huizhou"/>
<city name="东莞" city_pinyinname="dongguan"/>
<city name="珠海" city_pinyinname="zhuhai"/>
<city name="中山" city_pinyinname="zhongshan"/>
<city name="江门" city_pinyinname="jiangmen"/>
<city name="佛山" city_pinyinname="foshan"/>
<city name="肇庆" city_pinyinname="zhaoqing"/>
<city name="云浮" city_pinyinname="yunfu"/>
<city name="阳江" city_pinyinname="yangjiang"/>
<city name="茂名" city_pinyinname="maoming"/>
<city name="湛江" city_pinyinname="zhanjiang"/>
</citys>
</province>
<province name="贵州省" province_pinyinname="guizhou">
<citys>
<city name="贵阳" city_pinyinname="guiyang"/>
<city name="六盘水" city_pinyinname="liupanshui"/>
<city name="遵义" city_pinyinname="zunyi"/>
<city name="安顺" city_pinyinname="anshun"/>
<city name="毕节" city_pinyinname="bijie"/>
<city name="同仁" city_pinyinname="tongren"/>
</citys>
</province>
<province name="河北省" province_pinyinname="hebei" >
<citys>
<city name="石家庄" city_pinyinname="shijiazhuang"/>
<city name="邯郸" city_pinyinname="handan"/>
<city name="唐山" city_pinyinname="tangshan"/>
<city name="保定" city_pinyinname="baoding"/>
<city name="秦皇岛" city_pinyinname="qinhuangdao"/>
<city name="邢台" city_pinyinname="xingtai"/>
<city name="张家口" city_pinyinname="zhangjiakou"/>
<city name="承德" city_pinyinname="chengde"/>
<city name="沧州" city_pinyinname="cangzhou"/>
<city name="廊坊" city_pinyinname="langfang"/>
<city name="衡水" city_pinyinname="hengshui"/>
</citys>
</province>
<province name="黑龙江省" province_pinyinname="heilongjiang" >
<citys>
<city name="哈尔冰" city_pinyinname="haerbing"/>
<city name="齐齐哈尔" city_pinyinname="qiqihaer"/>
<city name="黑河" city_pinyinname="heihe"/>
<city name="大庆" city_pinyinname="daqing"/>
<city name="伊春" city_pinyinname="yichun"/>
<city name="鹤岗" city_pinyinname="hegang"/>
<city name="佳木斯" city_pinyinname="jiamusi"/>
<city name="双鸭山" city_pinyinname="shuangyashan"/>
<city name="七台河" city_pinyinname="qitaihe"/>
<city name="鸡西" city_pinyinname="jixi"/>
<city name="牡丹江" city_pinyinname="mudanjiang"/>
<city name="绥化" city_pinyinname="suihua"/>
</citys>
</province>
<province name="河南省" province_pinyinname="henansheng" >
<citys>
<city name="郑州" city_pinyinname="zhenzhou"/>
<city name="开封" city_pinyinname="kaifeng"/>
<city name="洛阳" city_pinyinname="luoyang"/>
<city name="平顶山" city_pinyinname="pingdingshan"/>
<city name="安阳" city_pinyinname="anyang"/>
<city name="鹤壁" city_pinyinname="hebi"/>
<city name="新乡" city_pinyinname="xinxiang"/>
<city name="焦作" city_pinyinname="jiaozuo"/>
<city name="濮阳" city_pinyinname="puyang"/>
<city name="许昌" city_pinyinname="xuchang"/>
<city name="漯河" city_pinyinname="luohe"/>
<city name="三门峡" city_pinyinname="sanmenxia"/>
<city name="南阳" city_pinyinname="nanyang"/>
<city name="商丘" city_pinyinname="shangqiu"/>
<city name="周口" city_pinyinname="zhoukou"/>
<city name="驻马店" city_pinyinname="zhumadian"/>
</citys>
</province>
<province name="湖北省" province_pinyinname="hubei" >
<citys>
<city name="武汉" city_pinyinname="wuhan"/>
<city name="十堰" city_pinyinname="shiyan"/>
<city name="襄阳" city_pinyinname="xiangyang"/>
<city name="荆门" city_pinyinname="jingmen"/>
<city name="孝感" city_pinyinname="xiaogan"/>
<city name="黄冈" city_pinyinname="huanggang"/>
<city name="鄂州" city_pinyinname="ezhou"/>
<city name="黄石" city_pinyinname="huangshi"/>
<city name="咸宁" city_pinyinname="xianning"/>
<city name="荆州" city_pinyinname="jingzhou"/>
<city name="宜昌" city_pinyinname="yichang"/>
<city name="随州" city_pinyinname="suizhou"/>
</citys>
</province>
<province name="湖南省" province_pinyinname="hunan" >
<citys>
<city name="长沙" city_pinyinname="changsha"/>
<city name="衡阳" city_pinyinname="hengyang"/>
<city name="张家界" city_pinyinname="zhangjiajie"/>
<city name="常德" city_pinyinname="changde"/>
<city name="益阳" city_pinyinname="yiyang"/>
<city name="岳阳" city_pinyinname="yueyang"/>
<city name="株洲" city_pinyinname="zhuzhou"/>
<city name="湘潭" city_pinyinname="xiangtan"/>
<city name="郴州" city_pinyinname="chenzhou"/>
<city name="永州" city_pinyinname="yongzhou"/>
<city name="邵阳" city_pinyinname="shaoyang"/>
<city name="怀化" city_pinyinname="huaihua"/>
<city name="娄底" city_pinyinname="loudi"/>
</citys>
</province>
<province name="吉林省" province_pinyinname="jinlin" >
<citys>
<city name="长春" city_pinyinname="changchun"/>
<city name="吉林" city_pinyinname="jinlin"/>
<city name="白城" city_pinyinname="baicheng"/>
<city name="松原" city_pinyinname="songyuan"/>
<city name="四平" city_pinyinname="siping"/>
<city name="辽源" city_pinyinname="liaoyuan"/>
<city name="通化" city_pinyinname="tonghua"/>
<city name="白山" city_pinyinname="baishan"/>
</citys>
</province>
<province name="江西省" province_pinyinname="jiangxi" >
<citys>
<city name="南昌" city_pinyinname="nanchang"/>
<city name="九江" city_pinyinname="jiujiang"/>
<city name="景德镇" city_pinyinname="jingdezhen"/>
<city name="鹰潭" city_pinyinname="yingtan"/>
<city name="新余" city_pinyinname="xinyu"/>
<city name="萍乡" city_pinyinname="pingxiang"/>
<city name="赣州" city_pinyinname="ganzhou"/>
<city name="上饶" city_pinyinname="shangrao"/>
<city name="抚州" city_pinyinname="fuzhou"/>
<city name="宜春" city_pinyinname="yichun"/>
<city name="吉安" city_pinyinname="jian"/>
</citys>
</province>
<province name="江苏省" province_pinyinname="jiangsu" >
<citys>
<city name="南京" city_pinyinname="nanjing"/>
<city name="徐州" city_pinyinname="xuzhou"/>
<city name="连云港" city_pinyinname="lianyungang"/>
<city name="宿迁" city_pinyinname="suqian"/>
<city name="淮安" city_pinyinname="huaian"/>
<city name="盐城" city_pinyinname="yancheng"/>
<city name="扬州" city_pinyinname="yangzhou"/>
<city name="泰州" city_pinyinname="taizhou"/>
<city name="南通" city_pinyinname="nantong"/>
<city name="镇江" city_pinyinname="zhenjiang"/>
<city name="常州" city_pinyinname="changzhou"/>
<city name="无锡" city_pinyinname="wuxi"/>
<city name="苏州" city_pinyinname="suzhou"/>
</citys>
</province>
<province name="辽宁省" province_pinyinname="liaoning" >
<citys>
<city name="沈阳" city_pinyinname="shenyang"/>
<city name="朝阳" city_pinyinname="chaoyang"/>
<city name="阜新" city_pinyinname="fuxin"/>
<city name="铁岭" city_pinyinname="tieling"/>
<city name="抚顺" city_pinyinname="fushun"/>
<city name="本溪" city_pinyinname="bexi"/>
<city name="辽阳" city_pinyinname="liaoyang"/>
<city name="鞍山" city_pinyinname="anshan"/>
<city name="丹东" city_pinyinname="dandong"/>
<city name="营口" city_pinyinname="yingkou"/>
<city name="盘锦" city_pinyinname="panjin"/>
<city name="锦州" city_pinyinname="jinzhou"/>
<city name="葫芦岛" city_pinyinname="huludao"/>
</citys>
</province>
<province name="山东省" province_pinyinname="shandong" >
<citys>
<city name="济南" city_pinyinname="jinan"/>
<city name="青岛" city_pinyinname="qingdao"/>
<city name="聊城" city_pinyinname="liaocheng"/>
<city name="德州" city_pinyinname="dezhou"/>
<city name="东营" city_pinyinname="dongying"/>
<city name="淄博" city_pinyinname="zibo"/>
<city name="潍坊" city_pinyinname="weifang"/>
<city name="烟台" city_pinyinname="yantai"/>
<city name="威海" city_pinyinname="weihai"/>
<city name="日照" city_pinyinname="rizhao"/>
<city name="临沂" city_pinyinname="linyi"/>
<city name="枣庄" city_pinyinname="zaozhuang"/>
<city name="济宁" city_pinyinname="jining"/>
<city name="泰安" city_pinyinname="taian"/>
<city name="莱芜" city_pinyinname="laiwu"/>
<city name="滨州" city_pinyinname="bingzhou"/>
<city name="菏泽" city_pinyinname="heze"/>
</citys>
</province>
<province name="陕西省" province_pinyinname="shangxi" >
<citys>
<city name="西安" city_pinyinname="xian"/>
<city name="延安" city_pinyinname="yanan"/>
<city name="铜川" city_pinyinname="tongchuan"/>
<city name="渭南" city_pinyinname="weinan"/>
<city name="咸阳" city_pinyinname="xianyang"/>
<city name="宝鸡" city_pinyinname="baoji"/>
<city name="汉中" city_pinyinname="hanzhong"/>
<city name="榆林" city_pinyinname="yulin"/>
<city name="商洛" city_pinyinname="shangluo"/>
<city name="安康" city_pinyinname="ankang"/>
</citys>
</province>
<province name="山西省" province_pinyinname="shanxi" >
<citys>
<city name="太原" city_pinyinname="taiyuan"/>
<city name="大同" city_pinyinname="datong"/>
<city name="朔州" city_pinyinname="shuozhou"/>
<city name="阳泉" city_pinyinname="yangquan"/>
<city name="长治" city_pinyinname="changzhi"/>
<city name="晋城" city_pinyinname="jincheng"/>
<city name="忻州" city_pinyinname="xinzhou"/>
<city name="吕梁" city_pinyinname="lvliang"/>
<city name="晋中" city_pinyinname="jinzhong"/>
<city name="临汾" city_pinyinname="linfen"/>
<city name="运城" city_pinyinname="yuncheng"/>
</citys>
</province>
<province name="四川省" province_pinyinname="sichuan" >
<citys>
<city name="成都" city_pinyinname="chengdou"/>
<city name="广元" city_pinyinname="guangyuan"/>
<city name="绵阳" city_pinyinname="mianyang"/>
<city name="德阳" city_pinyinname="deyang"/>
<city name="南充" city_pinyinname="nanchong"/>
<city name="广安" city_pinyinname="guangan"/>
<city name="遂宁" city_pinyinname="suining"/>
<city name="内江" city_pinyinname="neijiang"/>
<city name="乐山" city_pinyinname="leshan"/>
<city name="自贡" city_pinyinname="zigong"/>
<city name="泸州" city_pinyinname="luzhou"/>
<city name="宜宾" city_pinyinname="yibin"/>
<city name="攀枝花" city_pinyinname="panzhihua"/>
<city name="巴中" city_pinyinname="bazhong"/>
<city name="达州" city_pinyinname="dazhou"/>
<city name="资阳" city_pinyinname="ziyang"/>
<city name="眉山" city_pinyinname="meishan"/>
<city name="雅安" city_pinyinname="yaan"/>
</citys>
</province>
<province name="云南省" province_pinyinname="yunnan" >
<citys>
<city name="昆明" city_pinyinname="kunming"/>
<city name="曲靖" city_pinyinname="qujing"/>
<city name="玉溪" city_pinyinname="yuxi"/>
<city name="丽江" city_pinyinname="lijiang"/>
<city name="邵通" city_pinyinname="shaotong"/>
<city name="普洱" city_pinyinname="puer"/>
<city name="临沧" city_pinyinname="lincang"/>
<city name="保山" city_pinyinname="baoshan"/>
</citys>
</province>
<province name="浙江省" province_pinyinname="zhejiang" >
<citys>
<city name="杭州" city_pinyinname="hangzhou"/>
<city name="宁波" city_pinyinname="ningbo"/>
<city name="湖州" city_pinyinname="huzhou"/>
<city name="嘉兴" city_pinyinname="jiaxing"/>
<city name="舟山" city_pinyinname="zhoushan"/>
<city name="绍兴" city_pinyinname="shaoxing"/>
<city name="衢州" city_pinyinname="quzhou"/>
<city name="金华" city_pinyinname="jinhua"/>
<city name="台州" city_pinyinname="taizhou"/>
<city name="温州" city_pinyinname="wenzhou"/>
<city name="丽水" city_pinyinname="lishui"/>
</citys>
</province>
<province name="青海省" province_pinyinname="qinghai" >
<citys>
<city name="西宁" city_pinyinname="xining"/>
<city name="海东" city_pinyinname="haidong"/>
</citys>
</province>
<province name="海南省" province_pinyinname="hainan" >
<citys>
<city name="海口" city_pinyinname="haikou"/>
<city name="三亚" city_pinyinname="sanya"/>
<city name="文昌" city_pinyinname="wenchang"/>
<city name="儋州" city_pinyinname="danzhou"/>
<city name="东方" city_pinyinname="dongfang"/>
</citys>
</province>
<province name="广西省" province_pinyinname="guangxi" >
<citys>
<city name="南宁" city_pinyinname="nanning"/>
<city name="桂林" city_pinyinname="guilin"/>
<city name="柳州" city_pinyinname="liuzhou"/>
<city name="梧州" city_pinyinname="wuzhou"/>
<city name="贵港" city_pinyinname="guigang"/>
<city name="玉林" city_pinyinname="yulin"/>
<city name="钦州" city_pinyinname="qinzhou"/>
<city name="北海" city_pinyinname="beihai"/>
<city name="防城港" city_pinyinname="fangchenggang"/>
<city name="崇左" city_pinyinname="chongzuo"/>
<city name="百色" city_pinyinname="baise"/>
<city name="河池" city_pinyinname="hechi"/>
<city name="来宾" city_pinyinname="laibin"/>
<city name="贺州" city_pinyinname="hezhou"/>
</citys>
</province>
<province name="内蒙古" province_pinyinname="neimenggu" >
<citys>
<city name="呼和浩特" city_pinyinname="huhehaote"/>
<city name="包头" city_pinyinname="baotou"/>
<city name="乌海" city_pinyinname="wuhai"/>
<city name="赤峰" city_pinyinname="chifeng"/>
<city name="呼伦贝尔" city_pinyinname="hulunbeier"/>
<city name="通辽" city_pinyinname="tongliao"/>
<city name="鄂尔多斯" city_pinyinname="eerduosi"/>
</citys>
</province>
<province name="宁夏省" province_pinyinname="ningxia" >
<citys>
<city name="银川" city_pinyinname="yichuan"/>
<city name="石嘴山" city_pinyinname="shizuishan"/>
<city name="吴忠" city_pinyinname="wuzhong"/>
<city name="中卫" city_pinyinname="zhongwei"/>
<city name="固原" city_pinyinname="guyuan"/>
</citys>
</province>
<province name="西藏自治区" province_pinyinname="xizangzizhiqu" >
<citys>
<city name="拉萨" city_pinyinname="lasa"/>
</citys>
</province>
<province name="新疆维吾尔自治区" province_pinyinname="xinjiang" >
<citys>
<city name="乌鲁木齐" city_pinyinname="wulumuqi" />
<city name="克拉玛依" city_pinyinname="kelamayi"/>
</citys>
</province>
<province name="香港" province_pinyinname="xianggang" >
<citys>
<city name="香港" city_pinyinname="xianggang" />
</citys>
</province>
<province name="澳门" province_pinyinname="aomen" >
<citys>
<city name="澳门" city_pinyinname="aomen" />
</citys>
</province>
</provinces>
接着再在init包中创建一个recordPC.py文件，代码如下:
# -*- coding:utf-8 -*-
from xml.dom import minidom
from com.jhnet.spider.db.factory.MongoFactory import MongoFactory
def recordPc():
xmldoc = minidom.parse("province_city.xml")
ps = xmldoc.firstChild
p = ps.childNodes
for i in range(0, p.length):
pp = p.item(i)
if pp.nodeType == 1:
pname = pp.attributes['name'].value
ppinyin = pp.attributes['province_pinyinname'].value
ppCs = pp.childNodes.item(1)
ppC = ppCs.childNodes
for j in range(0, ppC.length):
ppCC = ppC.item(j)
if ppCC.nodeType == 1:
cname = ppCC.attributes['name'].value
cpinyin = ppCC.attributes['city_pinyinname'].value
MongoFactory.getDBCollection("province_city").insert({'province_name':pname,'province_pinyin':ppinyin,'city_name':cname,'city_pinyin':cpinyin})由于这个文件是依赖于mongodb的，所以创建完成之后就先不管了。
二，在factory包中创建MongoFactory.py文件，代码如下(因为是数据连接，索引代码就不用加注释了):
# -*- coding: utf-8 -*-
import pymongo
class MongoFactory:
__conn = None
__db = None
def __init__(self):
pass
def initMongo(self,config):
mainMongoHost = config['mainMongo_host']
mainMongodbName=config['mainMongo_dbName']
mainMongoConnectionsPerHost = int(config['mainMongo_connectionsPerHost'])
mainMongoConnectTimeout = int(config['mainMongo_connectTimeout'])
mainMongoPort = int(config['mainMongo_port'])
MongoFactory.conn = pymongo.Connection(mainMongoHost, mainMongoPort,mainMongoConnectionsPerHost,mainMongoConnectTimeout)
MongoFactory.db = MongoFactory.conn[mainMongodbName]
@staticmethod
def getConn():
return MongoFactory.conn
@staticmethod
def closeConn():
MongoFactory.conn.close()
@staticmethod
def closeDB():
MongoFactory.db.close()
@staticmethod
def getDBCollection(tableName):
return MongoFactory.db[tableName]
三，接下来在entity包中创建一个实体ProvinceCity.py,代码如下:# -*- coding: utf-8 -*-
class ProvinceCity:
def __init__(self,cityName, cityPinyin,provinceName,provincePinyin):
self.cityName = cityName
self.cityPinyin = cityPinyin
self.provinceName = provinceName
self.provincePinyin = provincePinyin
self.homeinnsCityid = None
def setCityName(self, cityName):
self.cityName = cityName
def getCityName(self):
return self.cityName
def setCityPinyin(self,cityPinyin):
self.cityPinyin = cityPinyin
def getCityPinyin(self):
return self.cityPinyin
def setProvinceName(self,provinceName):
self.provinceName = provinceName
def getProvinceName(self):
return self.provinceName
def setProvincePinyin(self, provincePinyin):
self.provincePinyin = provincePinyin
def getProvincePinyin(self):
return self.provincePinyin
def setHomeinnsCityid(self,homeinnsCityid):
self.homeinnsCityid = homeinnsCityid
def getHomeinnsCityid(self):
return self.homeinnsCityid
四，在util的__init__.py文件中放入如下代码:
# -*- coding:utf-8 -*-
from datetime import datetime
class Utils:
cityCache = []
"""
增加month
"""
@staticmethod
def getLastDate(incr):
dt = datetime.now()
y = int(dt.strftime('%Y'))
m = int(dt.strftime('%m'))
d = int(dt.strftime('%d'))
if m + incr > 12:
y = y + 1
m = m + incr - 12
d1 = datetime(y, m, d)
y = int(d1.strftime('%Y'))
m = int(d1.strftime('%m'))
d = int(d1.strftime('%d'))
tsm = None
if m < 10:
tsm = ('0' + str(m))
else:
tsm = str(m)
tsd =None
if d < 10:
tsd = ('0' + str(m))
else:
tsd = str(d)
nd = str(y)+"-"+str(tsm)+"-"+str(tsd)
return nd
@staticmethod
def getDate(incr):
dt = datetime.now()
y = int(dt.strftime('%Y'))
m = int(dt.strftime('%m'))
d = int(dt.strftime('%d'))
tsm = None
if m < 10:
tsm = ('0' + str(m))
else:
tsm = str(m)
tsd =None
if d < 10:
tsd = ('0' + str(d))
else:
tsd = str(d)
nd = str(y)+"-"+str(tsm)+"-"+str(tsd)
if m in [1,3,5,7,8,12]:
if (d + incr) > 31:
m += 1
d = (d + incr) - 31
else:
d += incr
elif m in [4,6,9,10,11]:
if (d + incr) > 30:
m += 1
d = (d + incr) - 30
else:
d += incr
else:
if Utils.isLeap(y):
if (d + incr) > 29:
m += 1
d = (d + incr) - 29
else:
d += incr
else:
if (d + incr) > 28:
m += 1
d = (d + incr) - 28
else:
d += incr
sm = None
if m < 10:
sm = ('0' + str(m))
else:
sm = str(m)
sd =None
if d < 10:
sd = ('0' + str(d))
else:
sd = str(d)
td = str(y) + "-" + str(sm) + "-" + str(sd)
return (nd,td)
@staticmethod
def isLeap(year):
if year%400 == 0:
return True
elif year%4 == 0 and year%100 == 0:
return True
五，在init包中创建InitConfig.py文件，代码如下:
# -*- coding:utf-8 -*-
#--------------------------------------------
#描述:如家页面爬虫主程序
#日期: 2014-02-09
#作者: jiangfuqiang
#---------------------------
from com.jhnet.spider.db.factory.MongoFactory import MongoFactory
from com.jhnet.spider.entity.ProvinceCity import ProvinceCity
from com.jhnet.spider.fetchpage.spider import SpiderUtils
from com.jhnet.spider.util import Utils
from xml.dom import minidom
import os
mongoConfigDict = {};
path = "E:"+os.sep+"testdata" + os.sep #写一个你自己的存放地址
#初始化mongodb的配置，放入字典中
def initMongoXml():
xmldoc = minidom.parse("../../../resources/mongodb-config.xml")
mongos = xmldoc.firstChild
mongoChild = mongos.childNodes
for i in range(0,mongoChild.length):
mongo = mongoChild.item(i)
if mongo.nodeType == 1: #去掉标签
mongoName = mongo.nodeName
mChild = mongo.childNodes
for j in range(0, mChild.length):
mm = mChild.item(j)
if mm.nodeType == 1:
mmName = mm.nodeName
mmValue = mm.childNodes.item(0).nodeValue
mongoConfigDict[mongoName + '_' + mmName] = mmValue
initMongoXml() #初始化mongodb配置
mf = MongoFactory()
mf.initMongo(mongoConfigDict)
#缓存城市
def cacheCity():
citys = MongoFactory.getDBCollection("province_city").find()
if citys.count() > 0:
for city in citys:
cityName = city['city_name']
cityNamePinyin = city['city_pinyin']
provincePinyin = city['province_pinyin']
provinceName = city['province_name']
pc = ProvinceCity(cityName, cityNamePinyin,provinceName,provincePinyin)
if 'homeinns_cityid' in city.keys():
pc.setHomeinnsCityid(city['homeinns_cityid'])
Utils.cityCache.append(pc)
cacheCity() #在这里你就可以执行recordPC.py了，赶紧先去初始化吧好了，今晚就先上这么多，明晚继续
本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。
