处理一个长度为 3000 万个字符的字符串
我正在从另一台服务器下载一个 CSV 文件作为来自供应商的数据源.
I am downloading a CSV file from another server as a data feed from a vendor.
我正在使用 curl 来获取文件的内容并将其保存到一个名为 $contents 的变量中.
I am using curl to get the contents of the file and saving that into a variable called $contents.
我可以很好地到达那部分，但是我尝试通过 和
进行爆炸以获取行数组，但它失败并显示 'out of内存错误.
I can get to that part just fine, but I tried exploding by and
to get an array of lines but it fails with an 'out of memory' error.
我 echo strlen($contents) 大约有 3050 万个字符.
I echo strlen($contents) and it's about 30.5 million chars.
我需要操作这些值并将它们插入到数据库中.我需要做什么来避免内存分配错误?
I need to manipulate the values and insert them into a database. What do I need to do to avoid memory allocation errors?
推荐答案
PHP 因内存不足而窒息.不要让 curl 用文件内容填充 PHP 变量，而是使用
PHP is choking because it's running out memory. Instead of having curl populate a PHP variable with the contents of the file, use the
CURLOPT_FILE
改为将文件保存到磁盘的选项.
option to save the file to disk instead.
//pseudo, untested code to give you the idea
$fp = fopen('path/to/save/file', 'w');
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_exec ($ch);
curl_close ($ch);
fclose($fp);
然后，一旦文件被保存，而不是使用 file 或 file_get_contents 函数(这会将整个文件加载到内存中，再次杀死 PHP)，使用 fopen 和 fgets 读取文件一行一次.
Then, once the file is saved, instead of using the file or file_get_contents functions (which would load the entire file into memory, killing PHP again), use fopen and fgets to read the file one line at a time.
相关文章