java - 分頁查詢中如何使用多線程加快處理速度?
問題描述
一個歷史表中有大量的數(shù)據(jù),現(xiàn)在要通過分頁式查詢處理轉(zhuǎn)換數(shù)據(jù)。
現(xiàn)在將處理數(shù)據(jù)的邏輯放在線程池中處理,以加快處理流程。
可是總是出現(xiàn)事務(wù)方面的異常
比如 : SQLNonTransientConnectionException
請問該如何解決上述異常,或者有什么好的多線程分頁查詢處理方案?
原來問題描述不太清楚,現(xiàn)在添加以下代碼(手敲,如果有錯,請多包含)
分頁式查詢邏輯:
int pageSize = 100;int currentPageLength = 0;int pageIndex = 0;ExecutorService exe = newFixedThreadPool(Runtime.getRuntime().availableProcessors());do { int offset = pageIndex * pageSize; List<TradeInfo> tradeInfos = tradeInfoService.findTradeInfoBysPage(queryParams,offset,pageSize); if (null != tradeInfos && tradeInfos.size() > 0) {currentPageLength = tradeInfos.size();TradeInfoProcesserTask task = new TradeInfoProcesserTask(tradeInfos );exe.execute(task);pageIndex++; }else{System.out.println('Page Query TradeInfo Got NOTHING! Break query loop!');break; }} while (currentPageLength == pageSize);exe.shutdown();while(true) { if(exe.isTerminated()){doOtherThings();System.out.println('分頁式多線程處理數(shù)據(jù)完畢!');break; }}
數(shù)據(jù)處理邏輯:
public class TradeInfoProcesserTask implements Runnable{ private volatile List<TradeInfo> tradeInfos;public TradeInfoProcesserTask (List<TradeInfo> _tradeInfos){tradeInfos = _tradeInfos; }@Override public void run() {processTradeInfos(); }private void processTradeInfos(){//do something with tradeInfos ..... }}
問題解答
回答1:邏輯先不說。現(xiàn)在沒有判斷多線程是否全部執(zhí)行完,while循環(huán)完就shutdown。。。將CountDownLatch通過構(gòu)造器傳入線程
ExecutorService exe = newFixedThreadPool(Runtime.getRuntime().availableProcessors());CountDownLatch latch = new CountDownLatch(?); //?代表開啟全部線程的數(shù)量do { int offset = pageIndex * pageSize; List<TradeInfo> tradeInfos = tradeInfoService.findTradeInfoBysPage(queryParams,offset,pageSize); if (null != tradeInfos && tradeInfos.size() > 0) {currentPageLength = tradeInfos.size();TradeInfoProcesserTask task = new TradeInfoProcesserTask(tradeInfos, latch);exe.execute(task);pageIndex++; }else{System.out.println('Page Query TradeInfo Got NOTHING! Break query loop!');break; }} while (currentPageLength == pageSize);latch.await(); //多線程全部執(zhí)行完exe.shutdown();doOtherThings();System.out.println('分頁式多線程處理數(shù)據(jù)完畢!');
public class TradeInfoProcesserTask implements Runnable{ private volatile List<TradeInfo> tradeInfos; private CountDownLatch latch;public TradeInfoProcesserTask (List<TradeInfo> _tradeInfos, CountDownLatch latch){tradeInfos = _tradeInfos;this.latch = latch; }@Override public void run() {processTradeInfos();latch.countDown(); }private void processTradeInfos(){//do something with tradeInfos ..... }}回答2:
分頁查詢是不并發(fā)(DAO)的,數(shù)據(jù)處理是并發(fā)(Service),所以你的事務(wù)級級別是設(shè)置在哪個環(huán)節(jié)呢?
相關(guān)文章:
1. android - 用textview顯示html時如何寫imagegetter獲取網(wǎng)絡(luò)圖片2. javascript - table列過多,有什么插件可以提供列排序和選擇顯示列的功能3. showpassword里的this 是什么意思?代表哪個元素4. javascript - windows下如何使用babel,遇到了困惑5. JavaScript事件6. python - 為什么正常輸出中文沒有亂碼,zip函數(shù)之后出現(xiàn)中文編程unicode編碼的問題,我是遍歷輸出的啊。7. javascript - js中向下取整8. android - rxjava多線程并發(fā)怎么控制順序9. 對mysql某個字段監(jiān)控的功能10. html - vue項目中用到了elementUI問題
