sparksql写入mysql出现死锁
来源:9-17 -需求三统计结果写入到MySQL
 
			慕前端5264115
2019-06-03
rdd.foreachPartition(rowIterator => {
val list = new ListBufferTrackingBasic
var dbName = ““
rowIterator.foreach(row => {
if (dbName.isEmpty) {
dbName = “serving_tracking_” + row._1
}
val dimensions = ArrayBufferInt
for (dim <- value.split(”,”)) {
dimensions.append(row._2.getAsInt)
}
val platform = row._2.getAsInt
val dmp_recognition = row._2.getAsLong
val imp_pv = row._2.getAsLong
val imp_uv = row._2.getAsLong
val clk_pv = row._2.getAsLong
val clk_uv = row._2.getAsLong
list.append(TrackingBasic(platform, dmp_recognition, imp_pv, imp_uv, clk_pv, clk_uv, dimensions: _*))
})
            //  将结果数据写入MYsql
            TrackingDao.insertTrackingBasic(dbName, tableName, tableColumns, list)
})
如上在foreach中组装list,然后调用dao的insert
try {
conn = DbUtil.getConnection()
conn.setAutoCommit(false) //  设置手动提交
        // 构建insert sql
        val sql = generateInsertSql(dbName, tableName, tableColumns)
        pstmt = conn.prepareStatement(sql)
        for (ele <- list) {
            val dimesions = ele.dimensions
            val dimLength = dimesions.length
            for (i <- Range(0, dimLength)) {
                pstmt.setInt(i + 1, dimesions(i))
            }
            pstmt.setInt(dimLength + 1, ele.platform)
            pstmt.setLong(dimLength + 2, ele.dmp_recognition)
            pstmt.setLong(dimLength + 3, ele.imp_pv)
            pstmt.setLong(dimLength + 4, ele.imp_uv)
            pstmt.setLong(dimLength + 5, ele.clk_pv)
            pstmt.setLong(dimLength + 6, ele.clk_uv)
            pstmt.addBatch()
        }
        pstmt.executeBatch()
        conn.commit()
    } catch {
        case e2: BatchUpdateException => {
            println("BatchUpdateException-->:    " + dbName + "." + tableName + "===>time:" + System.currentTimeMillis() + " Length:" +  e2.getUpdateCounts().length)
            println("BatchUpdateException-->Message:   " + e2.getMessage)
        }
        case e: Exception => {
            e.printStackTrace()
            println("jdbc-Exception->    ")
        }
    } finally {
        DbUtil.release(conn, pstmt)
    }
如下是捕捉到的异常,发生死锁
BatchUpdateException–>:    serving_tracking_917.tracking_basics_date_media_placement_ad_creative_province_city===>time:1559270145103 L
ength:665
BatchUpdateException–>Message:   Deadlock found when trying to get lock; try restarting transaction
1回答
- 
				  Michael_PK 2019-06-03 死锁这个没遇到过,不太清楚你的机器的情况。两个思路:1)代码问题 2)mysql环境问题 022019-06-03
相似问题
