php - affliate window csv to drupal node via batch api -

-

i have affliate window csv has 1 lac rows, want save nodes, , tried batch api.

but still getting php timeout error ..please help

function module_aw_batch(){     $operations = array();   $csv = file_directory_path().'/aw/datafeed_134642.csv';   $file = fopen($csv, 'r');   while (($data = fgetcsv($file)) !== false) {       $operations[] = array('module_aw_op', array($data));   }   $batch = array(     'title' => t('generating feeds'), // title display while running.     'operations' => $operations,     'finished' => 'module_aw_finished', // last function call.     'init_message' => t('importing...it may take 4-5 hours'),     'progress_message' => t('processed @current out of @total.'),     'error_message' => t('import feeds has encountered error.'),   );   batch_set($batch);   batch_process('admin/content/node/overview'); }

update (solved)

instead of reading whole csv file @ time, split csv read 5 lines per process

found solution

batch declaration

function modulename_batch(){     $operations[] = array('modulename_op', array());      $batch = array(         'title' => t('importing events'),         'operations' => $operations,         'finished' => 'modulename_finished',         'init_message' => t('message here'),         'progress_message' => 'message here',         'error_message' => t('message here'),     );     batch_set($batch);     batch_process('admin/content/node/overview');// redirect page }  function modulename_aw_op(&$context) {           if (!isset($context['sandbox']['offset'])) {         $context['sandbox']['offset'] = 0;         $context['sandbox']['records'] = 0;     }      $filename =file_directory_path().'/aw/datafeed_134642.csv'; // csv file      $fp = fopen($filename, 'r');      if ( $fp === false ) {     // failed open file         watchdog('modulename', 'failed open ' . $filename);         $context['finished'] = true;         return;     }      $ret = fseek($fp, $context['sandbox']['offset']);      if ( $ret != 0 ) {     // failed seek         watchdog('modulename', 'failed seek ' . $context['sandbox']['offset']);         $context['finished'] = true;         return;     }      $limit = 5;  // maximum number of rows process @ time     $done = false;      ( $i = 0; $i < $limit; $i++ ) {         $line = fgetcsv($fp);         if ( $line === false ) {             $done = true;             // no more records process             break;         }         else {                       $context['sandbox']['records']++;             $context['sandbox']['offset'] = ftell($fp);             $record = $context['sandbox']['records'];             // data          }     }      $eof = feof($fp);      if ( $eof )  {         $context['success'] = true;     }     $context['message'] = "processed " . $context['sandbox']['records'] . " records";      $context['finished'] = ( $eof || $done ) ? 1 : 0; }

Comments

Post a Comment

Popular posts from this blog

c# - SVN Error : "svnadmin: E205000: Too many arguments" -

-
i trying reposotries using c# code process svncommand = null; var psi = new processstartinfo("svnadmin"); psi.redirectstandardoutput = true; psi.redirectstandarderror = true; psi.useshellexecute = false; psi.arguments= @"dump c:\repositories\myrepo > c:\temp\myrepodumpfile.dump"; using (svncommand = process.start(psi)) { var myoutput = svncommand.standardoutput; var myerror = svncommand.standarderror; debug.write("output :" + environment.newline + environment.newline + myoutput.readtoend() + environment.newline + "error :" + environment.newline + environment.newline + myerror.readtoend()); svncommand.close(); } when use dump command commandline svnadmin dump c:\repositories\myrepo > c:\temp\myrepodumpfile.dump it works fine when try ...
Read more

c++ - Using OpenSSL in a multi-threaded application -

-
i have been writing soap client application in c++ on ubuntu using openssl https transport , pthreads threading. have number of threads - 1 central data acquisition thread periodically gets worker threads make soap requests via shared mutex protected queues. reading documentation openssl found is openssl thread-safe? in openssl faq describes mechanisms required ensure thread safety when using openssl. implemented , works fine. the reason question conceptual difficulty really. thinking implementing same functionality application has already, instead of using threads, create 2 seperate applications : 1 worker thread (of multiple copies running) , main data acquisition thread. use tcp sockets communicate between 2 rather mutex protected queues. may bad idea not important - confusing me would have implement same functions required ensure openssl thread safety in second approach or not? my guess not have , treated independent (indeed surely must many applications use openssl) re...
Read more

All overlapping substrings matching a java regex -

-
is there api method returns (possibly overlapping) substrings match regular expression? for example, have text string: string t = 04/31 412-555-1235; , , have pattern: pattern p = new pattern("\\d\\d+"); matches strings of 2 or more characters. the matches are: 04, 31, 412, 555, 1235. how overlapping matches? i want code return: 04, 31, 41, 412, 12, 55, 555, 55, 12, 123, 1235, 23, 235, 35. theoretically should possible -- there obvious o(n^2) algorithm enumerates , checks substrings against pattern. edit rather enumerating substrings, safer use region(int start, int end) method in matcher . checking pattern against separate, extracted substring might change result of match (e.g. if there non-capturing group or word boundary check @ start/end of pattern). edit 2 actually, it's unclear whether region() expect zero-width matches. specification vague, , experiments yield disappointing results. for example: string line = "xx90xx"; strin...
Read more