av一区二区在线观看_亚洲男人的天堂网站_日韩亚洲视频_在线成人免费_欧美日韩精品免费观看视频_久草视

您的位置:首頁技術文章
文章詳情頁

python - 多線程中mysql插入出錯

瀏覽:103日期:2022-06-28 16:07:14

問題描述

在寫爬蟲中,我要把爬取到的數據存到數據庫中.每一個頁面里邊有很多條目,比如一個人的訪客可能有很多個,于是插入卸載循環中,

try: sql_visitor=’INSERT INTO visitor (ownername,owneruid,visitorname,visitoruid,visittime) VALUE ('%s',%d,'%s',%d,'%s')’%(ownername,owneruid,visitorname,visitoruid,visitortime) print sql_visitor self.cursor.execute(sql_visitor) self.connect.commit()except Exception as e: print e

一個頁面一個線程,嫌棄慢的我開了5個

max_threads=5while uid < 8000000 or threadlist: for thread1 in threadlist:if not thread1.is_alive(): threadlist.remove(thread1) while len(threadlist) < max_threads and uid < 8000000:uid+=1thread2=threading.Thread(target=run,args=(uid,))thread2.setDaemon(True)thread2.start()threadlist.append(thread2) time.sleep(5)

運行很順利:

INSERT INTO visitor (ownername,owneruid,visitorname,visitoruid,visittime) VALUE ('huosai7',4893,'Liang2017',7252799,'2017-5-22 21:06')INSERT INTO personalinfo (ownername,owneruid,jifen,huajiao,xiaomijiao,jinbi,haoyou,zhuti,rizhi,xiangce,fenxiang,kongjianfangwenliang,youxiangyanzheng,shipinrenzheng,juzhudi,chushengdi,shangcifabiaoshijian,shangcihuodongshijian,zuihoufangwen,zhuceshijian,zaixianshijian,shengri,xingbie) VALUE('huosai7',4893,0,0,0,0,0,0,0,0,0,0,0,0,'','','2100-01-01 12:00','2100-01-01 12:00','2100-01-01 12:00','2004-1-3 19:28',0,'2100-01-01 12:00',0)INSERT INTO visitor (ownername,owneruid,visitorname,visitoruid,visittime) VALUE ('龍樂',4894,'Liang2017',7252799,'2017-5-22 21:06')(1062, 'Duplicate entry ’4894-7252799-2017-05-22 21:06:00’ for key ’PRIMARY’')INSERT INTO personalinfo (ownername,owneruid,jifen,huajiao,xiaomijiao,jinbi,haoyou,zhuti,rizhi,xiangce,fenxiang,kongjianfangwenliang,youxiangyanzheng,shipinrenzheng,juzhudi,chushengdi,shangcifabiaoshijian,shangcihuodongshijian,zuihoufangwen,zhuceshijian,zaixianshijian,shengri,xingbie) VALUE('龍樂',4894,0,0,0,0,0,0,0,0,0,0,0,0,'','','2100-01-01 12:00','2100-01-01 12:00','2100-01-01 12:00','2004-1-3 20:21',0,'2100-01-01 12:00',0).......

于是我將max_thread設置成10,于是結果如下:

INSERT INTO visitor (ownername,owneruid,visitorname,visitoruid,visittime) VALUE ('xiao61',4889,'Liang2017',7252799,'2017-5-22 21:06')

(2006, ’MySQL server has gone away’)

INSERT INTO personalinfo (ownername,owneruid,jifen,huajiao,xiaomijiao,jinbi,haoyou,zhuti,rizhi,xiangce,fenxiang,kongjianfangwenliang,youxiangyanzheng,shipinrenzheng,juzhudi,chushengdi,shangcifabiaoshijian,shangcihuodongshijian,zuihoufangwen,zhuceshijian,zaixianshijian,shengri,xingbie) VALUE('xiao61',4889,0,0,0,0,0,0,0,0,0,0,0,0,'','','2100-01-01 12:00','2100-01-01 12:00','2100-01-01 12:00','2004-1-3 15:56',0,'2100-01-01 12:00',0)

(2006, ’MySQL server has gone away’)

INSERT INTO visitor (ownername,owneruid,visitorname,visitoruid,visittime) VALUE ('糊涂酷酷熊',4897,'Liang2017',7252799,'2017-5-22 21:06')

(2006, ’MySQL server has gone away’)

INSERT INTO personalinfo (ownername,owneruid,jifen,huajiao,xiaomijiao,jinbi,haoyou,zhuti,rizhi,xiangce,fenxiang,kongjianfangwenliang,youxiangyanzheng,shipinrenzheng,juzhudi,chushengdi,shangcifabiaoshijian,shangcihuodongshijian,zuihoufangwen,zhuceshijian,zaixianshijian,shengri,xingbie) VALUE('糊涂酷酷熊',4897,611,0,1655,0,0,2,0,0,0,34,0,0,'','','2007-3-27 00:37','2007-3-27 00:37','2007-3-27 00:37','2004-1-3 21:08',0,'2100-01-01 12:00',1)

(2006, ’MySQL server has gone away’).......

可以看出2006出來了,然后我將max_thread設置成30,然后結果如下:

python - 多線程中mysql插入出錯

就將,夠詳細嗎?不夠詳細還需要什么只管說!

問題解答

回答1:

看這里,我猜你是用的是pymysql,它的線程安全描述為1,對應的pep249里面做了詳細的描述:

Threads may share the module, but not connections.

線程可以共享模塊但不能共享連接。這也就是說你可能得在每個線程中創建一個連接。

吶~為什么不用orm來做呢?

標簽: Python 編程
相關文章:
主站蜘蛛池模板: 国产成人精品免高潮在线观看 | 日韩精品一 | 五月天天色| 欧美日韩国产在线 | 在线日韩欧美 | 在线看一区二区三区 | 毛片日韩 | 中文字幕一区在线观看视频 | 国产精品成人一区二区三区 | 精品国产免费人成在线观看 | 97国产精品 | 极品粉嫩国产48尤物在线播放 | 欧美综合一区二区三区 | a级毛片国产 | 国产有码 | 午夜免费福利电影 | 亚洲人成人一区二区在线观看 | 久久在线精品 | 激情 一区 | 91免费在线播放 | 亚洲欧美一区二区三区国产精品 | 蜜桃毛片 | 超碰人人91 | 丁香婷婷综合激情五月色 | 中文字幕一区二区三区精彩视频 | 久久99这里只有精品 | 成年免费大片黄在线观看岛国 | 国产欧美一区二区三区久久人妖 | 日本 欧美 三级 高清 视频 | 亚洲一区不卡 | www.av7788.com| 欧美一级欧美三级在线观看 | 韩日在线视频 | 色av一区二区 | 国产一区二区三区久久久久久久久 | 丁香一区二区 | 国产中文区二幕区2012 | 欧美一区二区视频 | 欧美精品一区三区 | 国产小视频精品 | 欧美激情综合五月色丁香小说 |