Curl php parser (advance) Imvu parser 5 posts

OP Thu Mar 22, 2012 3:11 am
User avatar
  • Gender: Male
  • Posts: 45
  • Joined: Sep, 2011
  • Karma: 0
right guys here is what i was able to come up with its not done 100% but thought some of you would like to see where im at
Code
<?php
    function get_date_time($timestamp = 0)
   {

    if ($timestamp)

    return date("Y-m-d H:i:s", $timestamp);

    else
    return gmdate("Y-m-d H:i:s");         
   }
   
  class IMVU
  {

    
      private $contents = null;
       public function __construct()
      {                 
          $curl = curl::getInstance();
         
          try {
             
              $curl->setopt_array(array(
                      CURLOPT_URL     => 'http://www.imvu.com/catalog/web_mypage.php?user=uruserid&preview=1',
                      CURLOPT_RETURNTRANSFER => true,
                      CURLOPT_HEADER  => false,
                      CURLOPT_USERAGENT => 'Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)',
                      CURLOPT_REFERER => 'http://www.google.com/',
                      CURLOPT_TIMEOUT => '10'
              ));
             
              $this->contents = $curl->exec();
             
          } catch (Exception $e) {
              die($e->getMessage());
          }
      }


      public function scrape()
      {
          preg_match_all('/<table width="100%" border="0" cellspacing="0" cellpadding="0" id="aboutme_panel_body">/(.*?)<\/table>/si', $this->contents, $matches);
          if (count($matches[0]) == 1) {
             
              for ($i = 0; $i <= 1; $i++) {
                 
                  if (preg_match('|<b><span class="notranslate">(.*)</span></b>|U', $matches[0][$i], $nick)) {
                     
                      preg_match('/<font size="-1">Avatar since: <span class="notranslate">(.*)<\/span>/si', $matches[0][$i], $since);
                     
                      preg_match('|<i id="user_tagline"><span class="notranslate">(.*)</span>|U', $matches[0][$i], $tagline);
                     
                      preg_match_all('|<img width="160" height="220" src="(.*)"|U', $matches[0][$i], $image);
                     
                     
                      $pic   = (string)$image[1][0];   
                     
                      mysql_query("INSERT INTO `imvu` (`tagline`, `since`, `nick`)
                                   VALUES ('" . $tagline . "', '" . $since . "', '" . $nick . "')");
                     
                     
                     
                      if (!file_put_contents('imvu/pics/' . $nick . '.jpg', file_get_contents($pic))) {
                           throw new Exception('Couldn\'t write to imvu/pics directory.');
                      }
               
                  }
              }
          }
      }
  }




note make sure to edit this line
Code
CURLOPT_URL     => 'http://www.imvu.com/catalog/web_mypage.php?user=uruserid&preview=1',

replace uruserid with your user id i havent had a chance to test this yet but it should work ill be writing a way to disply it 2mor and also the db stuct for it
Thu Mar 22, 2012 4:57 pm
User avatar
Mafia Don
Legendary Family Member
  • Gender: Male
  • Posts: 12817
  • Joined: Nov, 2007
  • Advertisements: 407
  • Karma: 204
What is the intended output of this script?
OP Fri Mar 23, 2012 12:13 am
User avatar
  • Gender: Male
  • Posts: 45
  • Joined: Sep, 2011
  • Karma: 0
Code
mysql_query("INSERT INTO `imvu` (`tagline`, `since`, `nick`)
                                   VALUES ('" . $tagline . "', '" . $since . "', '" . $nick . "')");

as you can see in the qry there its storing the tagline from your imvu profile the avatar since date and the nick im going to make it where it can pull everything off there or some suggestion for what you/anybody wants it to pull let me know had a long day at work today so will try to get the code to use it and db stuct and all up 2morz morning
Fri Mar 23, 2012 12:58 pm
User avatar
Administrator
Legendary Family Member
  • Gender: Male
  • Posts: 6285
  • Joined: Jun, 2010
  • Advertisements: 45
  • Karma: 368
You think you could make a script that allows you to mass download ap and ga Icons? I wrote an icon spider in vb.net that can crawl imvu and download icons in bulk but I haven't been able to figure out how to write a login with it yet so it doesn't support ap icons. Or maybe update mine to support logging in. Here's the code: viewtopic.php?f=109&t=8061


As for ideas on what to make, maybe you could write a php equivalent to this javascript code and fix it for Spring: viewtopic.php?f=103&t=8074 I didn't know enough javascript to help him.
Fri Jan 03, 2014 1:14 pm
User avatar
  • Posts: 10
  • Joined: Jan, 2014
  • Karma: 0
I think I'll work on this one.
The trick should be to first submit a login form with AP enabled login data. As a response, the script should receive a cookie from IMVU. When crawling pages, the script can then send the cookie together with the request (manually, a PHP script won't do it automatically like a browser does by default), and access all pages that the user could see when logged in.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Sign in

Already have an account? Sign in here

SIGN IN NOW

Create an account

Sign up for a new account in our community. It's easy!

REGISTER A NEW ACCOUNT