Using cURL to get all links in a webpage

• May 7, 2010
• 0 Comment
• Guides
• Posted by Gee

The code below finds all links in the href attribute of anchor tags of a given page url.

[php]
<?
$request_url =’http://log.xmodx.com’;

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $request_url); // The url to get links from
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); // We want to get the respone
$result = curl_exec($ch);

$regex=’|<a.*?href="(.*?)"|’;
preg_match_all($regex,$result,$parts);
$links=$parts[1];
foreach($links as $link){
echo $link."<br>";
}
curl_close($ch);
?>

[/php]

• Tags: cURL, scrape