Using CURL for remote data manipulation

admin — Tue, 27/04/2010 - 12:09pm

Curl is powerful library of calls to get data from remote sites. Suppose you are developing a website for a local dealer of Honda. They might want you to fetch data related to the latest brands from the master website of Honda and display it in theirs. As long as this is legal, CURL is the way to go in getting this done.

<?php
/* Initiate CURL handler */
$curl_handle=curl_init();

/* URL of the page you want to fetch data */
curl_setopt($curl_handle,CURLOPT_URL,'http://fetchdatafromme.com/listingpage/');

/* Timeout the connection after 2 seconds. This helps to load the page faster */
curl_setopt($curl_handle,CURLOPT_CONNECTTIMEOUT,2);

/* Return the transfer as a string */
curl_setopt($curl_handle,CURLOPT_RETURNTRANSFER,1);

/* Execute and get the return data to a variable */
$data = curl_exec($curl_handle);

/* Close the connection and free system resources */
curl_close($curl_handle);

This is all you would need to safely fetch the data from the remote site. The data can then be parsed and displayed on your site as you want.

Now suppose, the page you are accessing is password protected and you have the access details. Then you would need to add the following line before execution.

curl_setopt($curl_handle, CURLOPT_USERPWD, $auth_name.':'.$auth_pass);

If there are values to be passed to the fetching page, to get the correct data , then you can add the following.

curl_setopt($curl_handle,CURLOPT_POST,true);
curl_setopt($curl_handle,CURLOPT_POSTFIELDS,$postFields);

If you wish to add headers to the output, add the following.

curl_setopt($curl_handle,CURLOPT_HEADER,true);
curl_setopt($curl_handle,CURLOPT_HTTPHEADER,$headers);

We can use CURL functions to upload files via ftp. Following is an example which illustrates this.

<?php

/* Create handler for the file to be ftp'd */
$fpath = '/folderpath/filename.ext';

$fhandle = fopen($fpath, "r");

/* Initiate CURL handler */
$curl_handle = curl_init();

/* URL, filename and access details of the FTP server */
curl_setopt($curl_handle, CURLOPT_URL, 'ftp://ftpuser:ftppass@ftpserver.com/uploadpath/newfilename.ext');

/* Setup curl to handle file upload */
curl_setopt($curl_handle, CURLOPT_UPLOAD, 1);

/* Set file transfer mode as ASCII */
curl_setopt($curl_handle, CURLOPT_TRANSFERTEXT, 1);

/* File to FTP */
curl_setopt($curl_handle, CURLOPT_INFILE, $fhandle);

/* Pass file size, in bytes, of the file to upload. */
curl_setopt($curl_handle, CURLOPT_INFILESIZE, filesize($fpath));

/* Return the transfer as a string */
curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, 1);

/* Execute and get the return data to a variable */
$returnval = curl_exec($curl_handle);

/* Close the connection and free system resources */
curl_close($curl_handle);

/* You may check the return value to see if the upload went through successfully */
print $returnval;
?>

Articles

Primary links

Using CURL for remote data manipulation

User login

Follow Us

Who's online

Who's new