Giter Site home page Giter Site logo

Comments (201)

JPH71 avatar JPH71 commented on May 28, 2024 2

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024 1

@j-ktz, here's the new repository. You are customer 00000001.

from pgma-modernized.

j-ktz avatar j-ktz commented on May 28, 2024 1

WOW. WOW. Okay, in my defense I did unzip it on my Mac but when I dragged the file to my NAS it rezipped it when it uploaded it over the network.

I don't mind a spanking though. Ha!

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

Hi Cody

I have made the changes to QueerClick, wayBig abd Fagalicious.
Tested QueerClick and WayBig all working fine
Fagalicious I am getting a 503 Service Unavailable error well before it gets to the changes I have made,
Could you check that your current fagalicious is working fine? Then try the new one?

All three:
If the height of the image is greater than the following ratios:

  • Posters Width:Height 1:1.5
  • Background Width:Height 16:9

The agents will first try and use Thumbor, if that fails and the operating system is windows it will use the vbscript at that only runs on windows.
We will need to create an apple script for MacOS and one for Linux systems to cater for those two.
If thumbor fails and the system is non-windows - it will use the original image size.... Sorry @j-ktz
I think you have a MAC

Here are the three bundles
QueerClick.bundle.zip
WayBig.bundle.zip
Fagalicious.bundle.zip

Happy testing @CodyBerenson

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

just checked the old fagalicious script.. works fine. will test the rest tomorrow. xoxoxoxo

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

@JPH71 The new Fagalicious bundle worked fine.

I noticed that you took requirement out of the preferences for a windows user to point the agent at its local install of PMS. So, it doesn't matter where the PMS was installed?

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

all three updated agents seem to work just fine. Yay!

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

Ok, so i've updated the master code with the three agents, and rewrote the readme to be specific to Jason's agents and to remove traces of Aiden.

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

Finally, I've left a bread crumb to this new repository on the existing (and I believe unsupported) master site that is what folks get a hit on when they google Plex Agents: LGBT-PlexPlugins/plex-gay-metadata-agent#61 (comment)

We've built it....lets see if they come.

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

Unless you've made changes to any of the under bundles, we should be good (if you recall, you zipped up the latest and greatest for j-ktz recently).

I'm convinced that GitHub came down hard on Aiden because of the violations that screencaps would have caused. I'm guessing that in the nature of his work, keeping his GitHub presence is important to him...and he didn't need to be affiliated with any more strikes against him. I made sure that the images and the language in the readme are clean.

In a day or so I'll close this issue and open a new one for image cropping across other platforms. I'll have no way to test them, though.

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

j-ktz avatar j-ktz commented on May 28, 2024

Hi friends! Yeah, I can totally test. Just let me know what I have to do. Sorry still a newb.

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

@JPH71 every once in a blue moon I have file that GEVI won't match.... interested in troubleshooting? i'm OK that the agent works almost always and we don't need perfection. Let me know your thoughts...

xoxo

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

Is that a Harry Potter reference?

LOL.

Here are three. Not sure if the word "The" is throwing (gevi won't allow you to use the word "The" when manually searching)...

The Apprentice 1 and 2, both by Delta Productions
The Auto Files 2 by Rudebox Media

And NO Proof of the Pudding Images! LOL LOL LOL

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

Here's another, also beginning with "The" and containing a Number:
The First Time 2 by XXX-Project

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

NO ESC! :(

HEre's another "the", minus a movie number.

(Arena Entertainment) - The Sauna (2003).mp4

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

if you were a red neck american, it would have been toiletpaper as priority #1

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

lordie. first i lost aiden (would we qualify that as a loss in fact?)...and now i've lost my most fragile and imperfect Jason. this is why we can't let the far east into ESC. unless they are going to buy me better vpn

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

FYI, my little unicorn, Aiden's thumbor instance (https://cdn.vigue.me/unsafe/0x0:800x1200/url.jpg) is back online. Not suggest we change a damn thing...just wanted you to know.

from pgma-modernized.

j-ktz avatar j-ktz commented on May 28, 2024

Hey everyone! I'm alive! Work has just been crazy this week, haven't had much free time. It's strange, now my agents are scraping ads and using those as thumbnails. Was that a bug that was fixed and a newer release of agents? Maybe I'm using an old one but every time I replace I have issues. It was the latest batch from the old threat. I'd post a screenshot but don't want to get anyone in trouble?

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

Hi j-ktz. The lastest agents are in the code tab for this repository.... Once or twice a week or two ago I had an ad from waybig show up as the poster. i ended up finding the same video on either fagalicious or queerclick, changed the name, rescanned the library, and it grabbed the correct poster from the different blog.

A quick fix when an agent doesn't grab a poster:

  1. in notepad or something similar, copy the link two messages about this
  2. copy the "image location" from the blog's poster
  3. replace url.jpg from #1 by pasting what you copied in #2
  4. copy the entire string
  5. use this as the URL for the poster. It should bring in a cropped poster from the blog directly into plex.

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

avatar commented on May 28, 2024

Code is written.

from pgma-modernized.

avatar commented on May 28, 2024

I have only used 6 of the 2000 captcha credits, so I would be happy to send you the URL and code.

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

avatar commented on May 28, 2024

I have. Fixing some last minute parsing issues. Should be done soon.

Also, the only studio names that matter (and DVDs should work, I haven't put in DVD specific scrapers yet, working on that after I post the code) are

Scenes and DVDs by Helix should have the studio name of Helix Studios in the filename
Scenes by Staxus should be Staxus
Scenes by 8TeenBoy should be 8TeenBoy
Scenes by CockyBoys should be CockyBoys.

Other than that, they do not matter. Also, year does not matter, and title does not matter (as long as it is basically original title from studio site) AKA it doesn't have to match with waybig title or fagalicious title.

from pgma-modernized.

avatar commented on May 28, 2024

plexmagic.zip

from pgma-modernized.

avatar commented on May 28, 2024

PHP script code for who is interested:

<?php
require("../vendor/autoload.php");
if($_GET['t']) {
ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);
}
use Symfony\Component\HttpClient\HttpClient;
use Intervention\Image\ImageManager;
header("Access-Control-Allow-Origin: *");
$query = $_GET['q'];
$studio = $_GET['studio'];

if($studio == "") {
    $response = array("error" => 1);
    echo json_encode($response);
}

$metadata = array("cast"=>array(),"images"=>array(), "bg"=>array(), "genres"=>array());

if(!isset($_GET['t']) && file_exists("saved/" . urlencode(str_replace("&","and",$query)) . ".json")) {
    header("Content-Type: application/json");
    echo file_get_contents("saved/" . urlencode($query) . ".json");
    die();
}

//STAXUS
if($studio == "Staxus") {
    $client = new \Goutte\Client();
    $urls = googleSearch("site:staxus.com+$query");
    $url = $urls[0];
    $crawler = $client->request('GET', $url);
    
    //Video Title
    $metadata["title"] = $crawler->filter("div[class='video-descr__title'] > div > div")->first()->text();
    //Video Description
    $metadata["summary"] = stripDiac($crawler->filter("div[class='video-descr__content']")->text());
    //Release Date
    $date = DateTime::createFromFormat('d/F/Y', explode(": ", trim($crawler->filter("div[class='video-details']")->first()->text()))[1]);
    $metadata["released"] = $date->format("d-m-Y");

    //Rating
    $metadata["rating"] = floatval(trim($crawler->filter("span[class='video-grade-average'] > strong")->text())) * 20;
    
    //Cast
    $crawler->filter("div[class='video-descr__model-item']")->each(function($node) {
        global $metadata;
        $url = $node->filter("div[class='thumb']")->attr("style");
        $url = explode("url('", $url)[1];
        $url = explode("'", $url)[0];
        $url = "https:" . $url;
        $name = $node->filter("p > a")->text();
        $member = array("image"=>cropHead($url), "name"=>$name, "role"=>"");
        array_push($metadata["cast"], $member);
    });
    //Images
    $crawler->filter("div[class='gallery-image col-md-4 col-sm-6 aspect-ratio'] > a")->each(function($node) {
        global $metadata;
        $url = $node->attr("style");
        $url = explode("url('", $url)[1];
        $url = explode("'", $url)[0];
        $url = str_replace("thumbs","1024watermarked", $url);
        $url = "https:" . $url;
        array_push($metadata["images"], "https://cdn.vigue.me/unsafe/" . $url);
    });
}

//8TeenBoy
else if($studio == "8TeenBoy") {
    $client = new \Goutte\Client();
    $url = "";
    $results = googleSearch("site:8teenboy.com+$query");
    foreach($results as $result) {
        if($url == "") {
            if(strpos($result, "video/") !== false) {
                $url = $result;
            }
        }
    }
    //echo $url;
    $crawler = $client->request('GET', $url);
    
    //Video Title
    $metadata["title"] = $crawler->filter("h2.pull-left")->first()->text();
    $metadata["title"] = stripDiac($metadata["title"]);
    //Video Description
    $metadata["summary"] = $crawler->filter("p[class='scene-description hide show-md']")->text();
    $metadata["summary"] = str_replace("’","'", $metadata["summary"]);
    //Release Date
    $metadata["released"] = "";
    $iafd = $client->request('GET', 'http://www.iafd.com/results.asp?searchtype=comprehensive&searchstring=' . urlencode($metadata["title"]));
    $iafd->filter("#titleresult > tbody > tr")->each(function($node) {
        global $query, $metadata, $client;
        $title = $node->filter("td")->eq(3)->text();
        if($title == "") {
            $title = $node->filter("td")->eq(0)->text();
        }
        $distro = $node->filter("td")->eq(2)->text();
        $metadata["title"] = str_replace("and","&",$metadata["title"]);
        if(strtolower($title) == strtolower($metadata["title"]) && $distro == "helixstudios.net") {
            $href = "http://www.iafd.com/" . $node->filter(".pop-execute")->first()->attr("href");
            $iafd = $client->request('GET', $href);
            try {
                $releaseDate = $iafd->filter(".biodata")->eq(8)->text();
                if($releaseDate != "No Data") {
                    $date = DateTime::createFromFormat('M d, Y', $releaseDate);
                    $metadata["released"] = $date->format("d-m-Y");
                }
            } catch(Exception $e) {

            }
            $metadata["title"] = $iafd->filter("h1")->first()->text();

            $nodes = $iafd->filter(".castbox");
            $nodes->each(function($node) {
                //echo "node";
                global $metadata;
                $name = trim($node->filter("a")->text());
                $src = $node->filter("img")->attr("src");
                $role = $node->text();
                $role = str_replace($name, "", $role);
                $role = trim($role);
                $role = urlencode($role);
                $role = str_replace("%C2%A0", "", $role);
                $member = array("name" => $name, "image" => $src, "role" => $role);
                array_push($metadata["cast"], $member);
            });
            return;
        }
    });
    //Rating
    $metadata["rating"] = 100;
    //Cast
    /*
    $crawler->filter("div[class='thumbnail-grid thumbnail-grid pure-g']")->first()->filter("div > div > a")->each(function($node) {
        global $metadata;
        $url = $node->filter("div[class='thumbnail'] > img")->attr("src");
        $url = str_replace("img/200w/","",$url);
        $url = str_replace("https", "http", $url);
        $url = cropHead($url);
        $name = $node->filter("div[class='thumbnail-bottom-text']")->text();
        $member = array("image"=>$url, "name"=>$name);
        array_push($metadata["cast"], $member);
    });
    */
    //Images
    $crawler->filter("img[src*='https://cdn.8teenboy.com/img/250h/media/stills/']")->each(function($node) {
        global $metadata;
        $url = $node->attr("src");
        $url = str_replace("img/250h/", "img/1920w/", $url);
        array_push($metadata["images"], "https://cdn.vigue.me/unsafe/" . $url);
    });
}

//Helix Studios
else if($studio == "Helix Studios") {
    $client = new \Goutte\Client();
    $query = str_replace("and","&",$query);
    $url = "";
    $results = googleSearch("site:helixstudios.net+$query");
    foreach($results as $result) {
        if($url == "") {
            if(strpos($result, "video") !== false || strpos($result, "movie") !== false) {
                $url = $result;
            }
        }
    }
    if(isset($_GET['url'])) {
        $url = $_GET['url'];
    }
    //echo $url;
    $crawler = $client->request('GET', $url);
    if(strpos($url, "HXM") !== false) {
        //DVD Title
        $metadata["title"] = $crawler->filter("div.boxContent > h3")->first()->text();
        //DVD Description
        $metadata["summary"] = $crawler->filter("p[class='description']")->text();
        $metadata["summary"] = str_replace("’","'", $metadata["summary"]);
        //Release Date
        $date = $crawler->filter("div.boxContent > div")->first()->text();
        $date = trim(explode(": ", $date)[1]);
        $date = DateTime::createFromFormat('F j, Y', $date);
        $metadata["released"] = $date->format("d-m-Y");

        //Rating
        $metadata["rating"] = 100;
        //Cast
        $crawler->filter("#scene-models > li")->each(function($node) {
            global $metadata;
            $url = $node->filter("a > img")->attr("src");
            $url = str_replace("/img/150w","",$url);
            $url = cropHead($url);
            $name = $node->filter("a > div")->text();
            $member = array("image"=>$url, "name"=>$name, "role"=>"");
            array_push($metadata["cast"], $member);
        });
        //Images
        $id = explode("/", explode("movie/", $url)[1])[0];
        array_push($metadata["images"], "https://cdn.vigue.me/unsafe/https://cdn.helixstudios.com/media/covers/".$id."_back_xlarge.1539280596.jpg");
        array_push($metadata["images"], "https://cdn.vigue.me/unsafe/https://cdn.helixstudios.com/media/covers/".$id."_front_xlarge.1539280596.jpg");
    } else {
        //echo $url;
        //Video Title
        $metadata["title"] = $crawler->filter(".scene-title")->text();
        //Video Description
        $metadata["summary"] = $crawler->filter("tr")->eq(1)->text();
        $metadata["summary"] = str_replace("’","'", $metadata["summary"]);
        //Release Date
        $html = $client->getResponse()->getContent();
        $html = explode("</td>", explode("Released: </span>", $html)[1])[0];
        $date = trim($html);
        $date = DateTime::createFromFormat('F j, Y', $date);
        $metadata["released"] = $date->format("d-m-Y");
        //Rating
        $metadata["rating"] = 100;
        //Cast
        $crawler->filter("tr")->first()->filter("a")->each(function($node) {
            global $metadata, $client;
            $url = $node->attr("href");
            $actorpage = $client->request('GET', $url);
            $url = $actorpage->filter("#modelHeadshot > img")->attr("src");
            $url = str_replace("img/320w/","",$url);
            $url = cropHead($url);
            $name = $node->text();
            $member = array("image"=>$url, "name"=>$name, "role"=>"");
            array_push($metadata["cast"], $member);
        });
        //Images
        $i = 0;
        $z = 0;
        $crawler->filter("img[src*='https://cdn.helixstudios.com/img/300h/media/stills/']")->each(function($node) {
            global $metadata, $i, $z;
            $url = $node->attr("src");
            $url = str_replace("img/300h", "img/1920w", $url);
            if($i < 10) {
                if($z % 2 != 1) {
                    array_push($metadata["images"],"https://cdn.vigue.me/unsafe/" .  $url);
                }
            }
            $i++;
            $z++;
        });
        $i = 0;
        $z = 0;
        $crawler->filter("img[src*='https://cdn.helixstudios.com/img/300h/members/stills/']")->each(function($node) {
            global $metadata, $i, $z;
            $url = $node->attr("src");
            $url = str_replace("img/300h", "img/1920w", $url);
            if($i < 10) {
                if($z % 2 != 1) {
                    array_push($metadata["images"],"https://cdn.vigue.me/unsafe/" .  $url);
                }
            }
            $i++;
            $z++;
        });
    }
    //file_put_contents("saved/" . urlencode($query) . ".json", json_encode($metadata));
}

//CockyBoys
else if($studio == "CockyBoys") {
    $client = new \Goutte\Client();
    $results = googleSearch("site:cockyboys.com+$query");
    $url = $results[0];
    $crawler = $client->request('GET', $url);
    
    //DVD Title
    $metadata["title"] = $crawler->filter("h1.sectionTitle")->first()->text();
    //DVD Description
    $metadata["summary"] = $crawler->filter(".movieDesc")->text();
    $metadata["summary"] = str_replace("’","'", $metadata["summary"]);
    //Release Date
    $date = $crawler->filter(".underPlayer > div")->eq(1)->filter("p > span")->first()->text();
    $date = trim(explode(": ", $date)[1]);
    $date = DateTime::createFromFormat('d/m/Y', $date);
    $metadata["released"] = $date->format("d-m-Y");

    //Rating
    $metadata["rating"] = floatval($crawler->filter(".underPlayer > div")->first()->filter("p")->text()) * 10;
    //Cast
    $crawler->filter(".movieModels > span")->each(function($node) {
        global $metadata;
        $url = $node->filter("a")->last()->filter("img")->attr("src");
        $url = preg_replace('/([^:])(\/{2,})/', '$1/', $url);
        $url = cropHead($url);
        $name = $node->filter("a")->first()->text();
        $member = array("image"=>$url, "name"=>$name, "role"=>"");
        array_push($metadata["cast"], $member);
    });
    $url = str_replace("?type=vids","?type=highres",$url);
    $crawler = $client->request('GET', $url);
    $crawler->filter(".thumbs")->each(function($node) {
        global $metadata;
        $url = $node->attr("src");
        $url = str_replace("thumbs","1024watermarked",$url);
        $url = preg_replace('/([^:])(\/{2,})/', '$1/', $url);
        array_push($metadata["images"], "https://cdn.vigue.me/unsafe/" . $url);
    });
}

//Combined Search
else {
    $client = new \Goutte\Client();
    $query = str_replace("and","&",$query);
    $results = googleSearch("site:waybig.com+OR+site:fagalicious.com+OR+site:bananaguide.com+$query");
    
    $urlG = "";
    foreach($results as $result) {
        if($urlG == "") {
            $url = $result;
            if(strpos($url, "ServiceLogin") !== false || strpos($url, "goToSite") !== false || strpos($url, "ftp.waybig.com") !== false || strpos($url, "https://www.waybig.com/gallery/") !== false || strpos($url, "https://www.waybig.com/blog/tag/") !== false || strpos($url, "https://www.waybig.com/tag/") !== false || strpos($url, "https://www.waybig.com/video/") !== false || strpos($url, "https://www.waybig.com/pornstars/") !== false) {
            } else {
                $urlG = $url;
            }
        }
    }

    $crawler = $client->request('GET', $urlG);
    $metadata["title"] = $query;
    if(strpos($urlG, "waybig") !== false) {
        //Waybig
        //echo "waybig";
        $slug = explode("/blog/", $urlG)[1];
        $year = explode("/",$slug)[0];
        $month = explode("/",$slug)[1];
        $day = explode("/",$slug)[2];
        $metadata["released"] = $day . "-" . $month . "-" . $year;

        //Poster
        $poster = $crawler->filter("img[src*='zing.waybig.com']")->first()->attr("src");
        $manager = new ImageManager(array('driver' => 'gd'));
        $url = $poster;
        $arrContextOptions=array(
            "ssl"=>array(
                "verify_peer"=>false,
                "verify_peer_name"=>false,
            ),
        );  
        $response = file_get_contents($url, false, stream_context_create($arrContextOptions));
        $filename = "zing-" . uniqid() . ".jpg";
        file_put_contents($filename, $response);
        $image = $manager->make(imagecreatefromjpeg($filename));
        $width = $image->width();
        $height = $width * 1.5;
        $poster = "https://cdn.vigue.me/unsafe/0x0:" . $width . "x" . $height . "/" . $poster;
        array_push($metadata["images"], $poster);
        unlink($filename);

        //IAFD infos
        $query = urlencode($query);
        $url = "https://google.com/search?q=site:iafd.com+$query+inurl:title.rme";
        $goog = $client->request("GET", $url);
        $url = $goog->filter("a[href*='/url']")->first()->attr("href");
        $url = explode("/url?q=", $url)[1];
        $url = explode("&", $url)[0];
        $url = urldecode($url);
        
        $iafd = $client->request("GET", $url);
        try {
            $releaseDate = $iafd->filter(".biodata")->eq(8)->text();
            if($releaseDate != "No Data") {
                $date = DateTime::createFromFormat('M d, Y', $releaseDate);
                $metadata["released"] = $date->format("d-m-Y");
            }
        } catch(Exception $e) {

        }
        $synopsis = trim($iafd->filter("#synopsis > .padded-panel")->first()->text());
        $metadata["summary"] = stripDiac($synopsis);

        //Cast
        $nodes = $iafd->filter(".castbox");
        $nodes->each(function($node) {
            //echo "node";
            global $metadata;
            $name = trim($node->filter("a")->text());
            $src = $node->filter("img")->attr("src");
            $role = $node->text();
            $role = str_replace($name, "", $role);
            $role = trim($role);
            if(strpos($role, "Credited") !== false) {
                $role = explode(")", $role)[1];
            }
            $role = urlencode($role);
            $role = str_replace("%C2%A0", "", $role);
            $member = array("name" => $name, "image" => $src, "role" => $role);
            array_push($metadata["cast"], $member);
        });
    } else if(strpos($urlG, "bananaguide") !== false) {
        //Posters
        $nodes = $crawler->filter("a[rel='gallery-image']");
        $nodes->each(function($node) {
            global $metadata;
            $url = $node->attr("href");
            $url = "https://bananaguide.com" . $url;
            array_push($metadata["images"], "https://cdn.vigue.me/unsafe/" .  $url);
        });

        //IAFD infos
        $query = urlencode($query);
        $url = "https://google.com/search?q=site:iafd.com+$query+inurl:title.rme";
        $goog = $client->request("GET", $url);
        $url = $goog->filter("a[href*='/url']")->first()->attr("href");
        $url = explode("/url?q=", $url)[1];
        $url = explode("&", $url)[0];
        $url = urldecode($url);
        
        $iafd = $client->request("GET", $url);
        try {
            $releaseDate = $iafd->filter(".biodata")->eq(8)->text();
            if($releaseDate != "No Data") {
                $date = DateTime::createFromFormat('M d, Y', $releaseDate);
                $metadata["released"] = $date->format("d-m-Y");
            }
        } catch(Exception $e) {

        }
        $synopsis = trim($iafd->filter("#synopsis > .padded-panel")->first()->text());
        $metadata["summary"] = stripDiac($synopsis);

        //Cast
        $nodes = $iafd->filter(".castbox");
        $nodes->each(function($node) {
            //echo "node";
            global $metadata;
            $name = trim($node->filter("a")->text());
            $src = $node->filter("img")->attr("src");
            $role = $node->text();
            $role = str_replace($name, "", $role);
            $role = trim($role);
            if(strpos($role, "Credited") !== false) {
                $role = explode(")", $role)[1];
                $role = trim($role);
            }
            $role = urlencode($role);
            $role = str_replace("%C2%A0", "", $role);
            $member = array("name" => $name, "image" => $src, "role" => $role);
            array_push($metadata["cast"], $member);
        });
    } else{
        //Site Title
        $sTitle = stripDiac($crawler->filter(".entry-title")->first()->text());
        //Release Date
        $date_raw = trim($crawler->filter(".meta-date")->text());
        $date = DateTime::createFromFormat('F j, Y', $date_raw);
        $metadata["released"] = $date->format("d-m-Y");

        //Cast
        $tags = $crawler->filter(".post-meta > a[href*='/tag/']");
        $tags->each(function($node) {
            global $sTitle, $metadata;
            $tag = trim(stripDiac($node->text()));
            if(strpos(strtolower($sTitle), strtolower($tag)) !== false && strpos($tag, " ") !== false) {
                //echo $tag;
                //tag in title, assume cast member?
                
                $img = getIAFDActorImage($tag);
                if($img != "") {
                    $member = array("name"=>$tag, "image"=>$img);
                    array_push($metadata["cast"], $member);
                }
                
            } else {
                //genre!
                array_push($metadata["genres"], $tag);
            }
        });

        //Poster
        array_push($metadata["images"], "https://cdn.vigue.me/unsafe/" . $crawler->filter(".mypicsgallery")->first()->filter("a > img")->first()->attr("data-src"));

        //Summary
        $summaryNodes = $crawler->filter(".entry-content > p");
        $metadata["summary"] = "";
        $summaryNodes->each(function($node) {
            global $metadata;
            $metadata["summary"] .= $node->text();
        });
        $metadata["summary"] = stripDiac($metadata["summary"]);
    }
    //file_put_contents("saved/" . urlencode($query) . ".json", json_encode($metadata));
}

if($metadata["title"] != "") {
    $i = 0;
    file_put_contents("saved/" . urlencode(str_replace("&","and",$_GET['q'])) . ".json", json_encode($metadata));
    if($_GET['t'] == 1) {
        echo str_replace("\\","",json_encode($metadata));
    } else {
        header("Content-Type: application/json");
        echo json_encode($metadata);
    }
}

function minimum($b,$s) {
    if($b-$s < 0) {
        return 0;
    } else {
        return $b - $s;
    }
}
function getIAFDActorImage($name) {
    $name = strtolower($name);
    sleep(2);
    $q = "iafd " . $name . " inurl:person.rme";
    $url = "https://google.com/search?q=" . urlencode($q);
    $client = new \Goutte\Client();
    $goog = $client->request("GET", $url);
    $url = $goog->filter("a[href*='/url']")->first()->attr("href");
    $url = explode("/url?q=", $url)[1];
    $url = explode("&", $url)[0];
    $url = urldecode($url);
    $final = "";
    $genders = array("m","d");
    foreach ($genders as $gender) {
        try {
            $iafd = $client->request("GET", $url);
            $img = $iafd->filter("#headshot > img")->first()->attr("src");
            if(strpos($img, "nophoto340.jpg") !== false) {
                $final = "";
                break;
            }
            $final = $img;
        } catch(Exception $e) {
            $final = "";
            break;
        }
    }
    return $final;
}
function cropHead($url, $padding = 50) {
    $urls = 'https://neural.vigue.me/facebox/check';
    $data = array('url' => $url, 'faceprint' => 'false');

    // Setup cURL
    $ch = curl_init($urls);
    curl_setopt_array($ch, array(
        CURLOPT_POST => TRUE,
        CURLOPT_RETURNTRANSFER => TRUE,
        CURLOPT_HTTPHEADER => array(
            'Content-Type: application/json',
            'Accept: application/json'
        ),
        CURLOPT_POSTFIELDS => json_encode($data)
    ));

    // Send the request
    $result = curl_exec($ch);
    $json = json_decode($result);

    if($json->facesCount == 1) {
        $face = $json->faces[0]->rect;
        $crop = "";
        $crop .= minimum($face->left, $padding) . "x" . minimum($face->top, $padding) . ":";
        $crop .= (($face->left + $face->width) + $padding) . "x" . (($face->top + $face->height) + $padding);
        return "https://cdn.vigue.me/unsafe/" . $crop . "/" . $url;
    } else {
        return $url;
    }
}
function stripDiac($text) {
    $text = str_replace("”","\"", $text);
    $text = str_replace("“","\"", $text);
    $text = str_replace("‘","'", $text);
    $text = str_replace("’","'", $text);
    $text = str_replace("–","-", $text);
    $text = str_replace("&","and", $text);
    $text = str_replace("…","...", $text);
    return $text;
}
function posterOrBg($url) {
    $client = new \FasterImage\FasterImage();
    $images = $client->batch([
        $url
    ]);
    
    foreach ($images as $image) {
        //print_r($image['size']);
        if($image["size"][0] >= $image["size"][1]) {
            //wandscape
            return "bg";
        } else {
            return "poster";
        }
    }
}
function googleSearch($query) {
    $response = file_get_contents("https://vigue.me/api/googleSearch.php?q=" . urlencode($query));
    $results = json_decode($response);
    return $results;
}
?>

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

avatar commented on May 28, 2024

Magic.bundle makes a GET request to the magic-metadata.php file hosted on my site. That script calls googleSearch.php which uses a selenium grid instance virtualized in docker to scrape Google using an actual browser to not get blocked. All images are proxied through cdn.vigue.me to cache for 1 week to speed up future scrapes for the same title.

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

Terrific to see you back Aiden. Stay healthy everyone!

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

Hi Thanks for the innovative approach to Agent scraping.

I've tested against the seven most recent additions to WayBig, with varied results:

Matched, retrieves mostly accurate metadata (see below), retrieved posterart:

  1. (TimTales) - Tim Kruger and Santiago Rodriguez (2020) although posterart is correct, scene description, release date, etc. is mismatched
  2. (Corbin Fisher) - Ethan IV (2020) although posterart is correct, scene description, release date, etc. is mismatched
  3. (Fuckermate) - Abel Sanztin and Valdo Smith (2020) all metadata is accurate

Matched, retrieved mostly accurate metadata (release date is incorrect), retrieved NO posterart:

  1. (CockyBoys) - Calvin Banks and Nico Leon (2020)

No Match:

  1. (Active Duty) - Jesse Nice and Brandon Anderson (2020)
  2. (HotHouse) - Arad Winwin and Angel Rivera (2020)
  3. (Next Door Casting) - Rockey Goldenrod (2020) with or without spaces in studio name

Hope this helps.

Cody
com.plexapp.agents.Magic.log

from pgma-modernized.

avatar commented on May 28, 2024

The No match ones are due to incomplete IAFD pages or mismatched results, I will add in checks and the sort to verify data before trying alternate sources. Thank you for the heads up!

I think the reason my library was scraped so well, was most of it was Helix & 8TeenBoy with only about 300-400 titles coming from WayBig.

Also, could you please take my name out of the comments, I'm converting this account to be a specific Github account for this project.

Thanks again!

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

@JPH71 please check in, let us know you and yours are doing ok.

from pgma-modernized.

avatar commented on May 28, 2024

Google has no search filter for what I am trying to accomplish. I might make it so that it first allows you to manually select a result to use when auto matching doesn't work. Sorry I forgot to respond, I am currently transcoding a 48TB RAID array from H.264 to HEVC to save space as it is full. Fun times

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

j-ktz avatar j-ktz commented on May 28, 2024

@JPH71 It's been doing it for Waybig for me. Where can I find the newest agent where this is fixed?

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

@j-ktz and @JPH71- the bundles that Jason posted above (now 10 days ago) were uploaded into the master immediately after all three were tested, the next day (now 9 days ago). Please archive your current bundles and retrieve the latest bundles from the code tab. The code tab always has the latest updated and tested bundles.

I would suggest to change the image cropping Thumbor preferences for each of the three scene bundles to:

https://cdn.vigue.me/unsafe

image

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

p.s. sorry if that sounded terse. it was NOT my intention. My intention is only to be helpful.

Cheers!

from pgma-modernized.

j-ktz avatar j-ktz commented on May 28, 2024

@CodyBerenson Ty, Cody!!! I got 'em all updated. There was a couple that I refreshed that still pulled ads and not the cover art, but I manually modified them. I hope you both are staying safe with all this craziness going on.

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

@j-ktz If you wouldn't mind, would you tell me the name (no screan caps!) of the clip(s)? I'd like to test it out on my end. The code very well could need a new tweak if WayBig has made any changes, again. Thanks!

Hope you too are staying healthy, thank you. The highlight of my leaving the house was to take the recycle bins to the curb first thing this a.m., and then retrieve them an hour ago. Woohoo!!!

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

@JPH71 I was having problems with the posters not returning from all of the agents, so i updated the preference on each of the agents back to https://cdn.vigue.me/unsafe

Ever since, posters return fine. May want to consider.

from pgma-modernized.

j-ktz avatar j-ktz commented on May 28, 2024

@CodyBerenson I don't recall which titles didn't pull in posters but I have the new agents now and tested 2 new movies and both worked great. If I have more issues with the posts, I'll let you know which titles they are. Thanks!!!

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

ok, sounds good.

at the time that posters weren't returning, I didn't say anything because i was also having proxy issues so thought it may have something to do with that. when i would manually query the http://34.67.235.246:8888/unsafe/ thumbor instance, it would return the cover just fine, so i was convinced it was the proxy thing since the VB script didn't fire off and return the poster. then when @j-ktz was having his issue, I changed my preference back to the https://cdn.vigue.me/unsafe thumbor instance and everything has worked perfectly since...no more not returning posters.

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

j-ktz avatar j-ktz commented on May 28, 2024

Hey friends! Recently moved my Plex server to a new NAS. 70%+ of my posters didn't come over. All the other metadata is there, but not a lot of the posters. I've tried refreshing metadata for the whole library, items individually, emptying trash and re-scanning but no dice. Any other ideas?

Any new items I add, pull in posters perfectly, it's all my other items.

screenshot 2020-03-28 at 5 18 09 PM

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

hi @JPH71

are you able to see my repositories? if so, you can see the WayBig history here:

https://github.com/CodyBerenson/pgma/commits/master/WayBig.bundle

You should be able to grab the code from january.

I also found an archive from january, not sure its what you're looking for
01272020_WayBig.bundle.zip

And an archive from early february, again, not sure its what you're looking for
02092020_WayBig.bundle.zip

Don't forget in your non existent spare time to see if you can figure out the problem with files starting with "The" in GEVI #1 (comment) and #1 (comment)

Miss you. Be healthy and safe.

xoxo

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

j-ktz avatar j-ktz commented on May 28, 2024

@JPH71 Hmm, I've tried that, still no luck. Bummer :/

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

For what its worth....personally, I've never successfully migrated a PMS library. There are tons of faqs on the plex forums on how to do it, and i followed them to a 'T'...and it just never worked. Perhaps because i was not migrating across a similar OS (they warn about that in the Plex forums)....it just never worked. It was easy enough to recreate my movies and tv libaries, the built in plex agents are fantastic about that. i never got the adult libraries to migrate....so, i ended up moving the content to the NAS and leaving the PMS on the original OS. Moving the content and leaving the PMS in place works flawlessly. Just my experience and observations. Alas.

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

@JPH71 that was my first thought also (is he using the proper agent) but, he said that new content is indexing correctly so that's probably not the case.

I think he's going to have to individually re-match. mass 'refresh metadata' has been hit or miss.

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

@JPH71 the three blog agents are working perfectly for me. i am not sure what problem(s) you are having that you are fixing, so not sure what i will be testing or opening new threads for...

i wonder if the coconut has finally cracked...? lol

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

ok that should be easy to test. ill put in a bad thumbor address

re the timtales valentines day scene (assuming thats the one)....if one follows the rule of changing all curly single quotes to proper apostrophes that hot scene matched perfectly for me, first attempt.

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

j-ktz avatar j-ktz commented on May 28, 2024

So I just tried to Fix Match and it says no matches, even though the naming is incorrect (and it was matched in the past with the same name) and it's using the Gay Adult agents. Most of my items come up with no matches. Any ideas?

screenshot 2020-03-30 at 8 13 09 AM

screenshot 2020-03-30 at 8 12 59 AM

screenshot 2020-03-30 at 8 12 42 AM

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

@j-ktz a couple questions:

  1. can you attach your log?

  2. since moving to your new setup, have you had anything properly match, that wasn't already matched on your old setup?

  3. have you tried to first unmatch and then fix match?

  4. what OS is your new PMS?

I'm going to share my own experience, probably completely unrelated but what the heck. when i expanded my NAS, i decided to consolidate all my adult stuff on the NAS. I did a ton of research and methodically took a couple step approach. First I moved the content leaving the existing PMS in place. As I moved the content to the NAS and rescanned, the existing PMS found the new content and updated its file location. Terrific! Then, I attempted to migrate my PMS. Nothing worked. in my case, I migrated from Windows to Freenas (FreeBSD). I had already migrated our Movies and TV libraries (as I said earlier, I had to totally recreate the libraries). However, re: the adult stuff, I couldn't get new stuff to scan, I couldn't get old stuff to get recognized. I did a bunch of research and found out that on the OS that I moved to, there are native issues with sharing/privileges for non-Plex branded agents. I followed each of the FAQs I could find about changing privileges. Nothing worked. I ended up keeping my old windows PMS, with the content now consolidated on the NAS. So, we have two PMSs in our home: 1. with all our movies, tv shows, music, family movies, etc. 2. with the adult stuff.

Again, just my experience but wanted to share.

from pgma-modernized.

j-ktz avatar j-ktz commented on May 28, 2024

@CodyBerenson

  1. Logs attached! If you can find anything out by taking a look that would be awesome.
  2. Movies and TV are matching fine. New porn I add works flawlessly. It's all my content content.
  3. Yes, if I try to match old content using the agents, it doesn't work, unless I chose Personal Media agent, but if I select it, poster doesn't come in.
  4. I'm using a Synology NAS.

Thanks for sharing your experience! Just frustrating that new content I add is beautiful and works great, and all my old stuff is blank. :/

Archive.zip

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

j-ktz avatar j-ktz commented on May 28, 2024

Hi @JPH71,

Thanks for the quick response. I have a Porn library (separate from TV and movies) that has mostly scenes, and a few films. I have all agents in my plugins folder and they're all checked off. I've already tried restarting the server too... not sure why it isn't working.
screenshot 2020-03-30 at 11 53 24 AM

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

j-ktz avatar j-ktz commented on May 28, 2024

@JPH71 Did what you said ,and tried to refresh again. It's strange, it'll refresh items and skip some others. It's like it thinks there is a poster? When I go to Get Info, and look at poster, it's like it has something selected but nothing is there.
screenshot 2020-03-30 at 12 30 15 PM

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

@j-ktz , @CodyBerenson
Here is the latest waybig....

bundle removed

from pgma-modernized.

j-ktz avatar j-ktz commented on May 28, 2024

Thanks @JPH71 . So weird... I shut down PMS, replaced the Waybig bundle, and restarted and now it doesn't appear under any agents?
screenshot 2020-03-30 at 1 00 47 PM
screenshot 2020-03-30 at 1 00 30 PM
screenshot 2020-03-30 at 1 00 25 PM

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

@JPH71 what am i looking for....just that the fallback vbs fires off, or also about the single curly quotes?

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

ok, this should be an easy set of tests. fyi, it was only WayBig that had the curly apostrophe problem. Fag and QC work with about any string you throw at it... will advise on my testing in a bit

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

Sorry @JPH71. I put in a bogus thumbor address. I took the 6 most recent WayBig blogs and changed nothing (usually i would change every single quote to an apostrophe).

1 had no curly quote, it matched fine, and returned a VBS cropped poster. Success
5 had multiple curly quotes. None matched:

(Lucas Entertainment) - Drake Rogers and Dante Lauro Service Andrey Vic’s Uncut Cock in ‘Banging Ruslan’s Bubble Butt, Scene Two’ (2020).mp4

(MEN Bareback) - Ryan Bones Fucks Aiden Ward in ‘Dickmatized’ (2020).mp4

(Next Door STUDIOS) - Aspen and Dalton Riley Tag-Team Elliot Finn in ‘My Boyfriend’s Step Brother’ (2020).mp4

(Next Door Studios) - Collin Simpson Fucks Nic Sahara in ‘My Step Bro’s Secret Member Fantasy’ (2020).mp4

(Next Door TABOO) - Roman Todd Fucks Lance Ford in ‘Teaching My Step Brother How To Ride’ (2020).mp4

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

Before your fix, i would have corrected three curly quotes to apostrophes in the following:

(Lucas Entertainment) - Drake Rogers and Dante Lauro Service Andrey Vic’s Uncut Cock in ‘Banging Ruslan’s Bubble Butt, Scene Two’ (2020).mp4

Vic’s to Vic's
‘Banging to 'Banging
Ruslan’s to Ruslan's

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

oh, then i was confused. lol. I thought you were addressing the curly single quotes that made your TimTales file fail. So the curly single quotes still need to be corrected. Roger. I'll give it a try

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

CodyBerenson avatar CodyBerenson commented on May 28, 2024

Hey Jason,

I corrected all of the single curlies, and nothing matches.

from pgma-modernized.

JPH71 avatar JPH71 commented on May 28, 2024

from pgma-modernized.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.