excel中数据插入数据库时出错
来源:3-7 excel导入数据到数据库

艾卡西亚在下雨
2019-04-14
在老师的指导下,已经成功的将爬取的数据写入了excel 格式如下:
1 王思聪吃玉米 3569222
2 宋慧乔签约王家卫公司 1992051
…
单条手动赋值插入正常
但是在数据批量插入数据库中时,会报错,可能是编码问题
import MySQLdb
from bs4 import BeautifulSoup
import requests
from openpyxl import Workbook, load_workbook
main_url = "https://s.weibo.com"
def get_list(url):
html_doc = requests.get(url).text
soup = BeautifulSoup(html_doc, features="lxml")
links = soup.select("tbody")[0].select('tr')
ranktop = [] #存放排名
name = [] #存放名称
hot_num = [] #存放热度
#print(links)
links = links[1:]
#print("{:<10}\t{:<20}\t\t{:<20}".format("排名", "名称", "热度"))
for tr in links:
#print(tr)
mz = tr.select('a')[0].contents[0].string
name.append(mz)
pm = int(tr.select('td')[0].string)
ranktop.append(pm)
rd = tr.select('span')[0].string
hot_num.append(rd)
wb = Workbook()
ws = wb.active
ws.title = u'微博热搜排行'
ws.append(['排名', '名称', '热度'])
for i in range(50):
l = [ranktop[i], name[i].encode('utf-8'), hot_num[i].encode('utf-8')]
ws.append(l)
wb.save('./static/weibohot_data.xlsx')
conn = get_conn()
ws = load_workbook('./static/weibohot_data.xlsx')
names = ws.get_sheet_names()
print(names)
wb = ws.active
for (i, row) in enumerate(wb.rows):
if i < 1:
continue
rank = wb['A{0}'.format(i + 1)].value
name = wb['B{0}'.format(i + 1)].value
hot_num = wb['C{0}'.format(i + 1)].value
# print(rank)
# print(name)
# print(hot_num)
cursor = conn.cursor()
sql = 'INSERT INTO `top_issue`( `rank`, `name`, `hot_num`) VALUES ({rank}, {name.encode(utf-8), {hot_num})'.format(
rank=rank, name=name, hot_num=hot_num)
print(sql)
cursor.execute(sql)
conn.autocommit(True)
def get_conn():
"""获取mysql的连接"""
try:
conn = MySQLdb.connect(
db='weibo_hot',
host='localhost',
user='root',
port=3366,
password='',
charset='utf8'
)
except:
print('connect failed')
return conn
def main():
url = "https://s.weibo.com/top/summary?cate=realtimehot"
get_list(url)
if __name__ == '__main__':
main()
由于是新手,有许多问题不懂,望老师见谅
写回答
1回答
-
NavCat
2019-04-14
format写法有误,试试这样:
sql = 'INSERT INTO `top_issue`( `rank`, `name`, `hot_num`) VALUES ({rank}, {name}, {hot_num})'.format( rank=rank, name=name.encode('utf-8'), hot_num=hot_num)
012019-04-14
相似问题