The website have links to different streaming sites online like youtube, openload, powvideos,....
Generally, due to copyright claims, these videos are deleted and as a result, this links will go to an SOFT 404 error page (the header response is 200 ok).
404 - examples
[login to view URL]
[login to view URL]
[login to view URL]
I think the easiest way to eliminate at least part of these dropped links is to create a script that does the following directly on the DB, with the help of a cronjob (to avoid the server overload 1.400.000 links on the DB):
1 - the script need to open external links from my DB (table "videos" column "url")
2 - if the rezult of the external page is 404 or soft 404, then change in the DB under column "approved" from 1 to "0"
3 - It would be fantastic if it could work for all sites, but if need custom check on each external website, at least, it must work for openload, streamplay and powvideos.
Suggestion - Because these pages return a standard title for all soft 404...
For example need to create some code to first check the page title.
All 404 pages from openload , for example, will have this page title :
<title>File not found ;(</title>
Others like powvideos, will have different title based on user language (english, spanish,....)
In the case that you see it possible, it is only necessary to add a configuration file, to which I can add the titles one by one later (need to be able to open this file from FTP) ...
The answer at this question, can give some idea ... [login to view URL]
An very OLD - Python library, [login to view URL] - [login to view URL]
This is the server data:
cPanel Version 78.0 (build 23)
Apache Version 2.4.39
PHP Version 7.0.33
MySQL Version 10.1.40-MariaDB
Operating System linux
Path to Perl /usr/bin/perl
Perl Version 5.16.3
Kernel Version 3.10.0-957.12.2.el7.x86_64
This is an small example of the table "videos" in my DB
CREATE TABLE `videos` (
`id` int(10) UNSIGNED NOT NULL,
`name` varchar(191) COLLATE utf8mb4_unicode_ci NOT NULL,
`thumbnail` varchar(191) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`url` varchar(191) COLLATE utf8mb4_unicode_ci NOT NULL,
`type` varchar(50) COLLATE utf8mb4_unicode_ci NOT NULL,
`quality` varchar(50) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`title_id` int(10) UNSIGNED NOT NULL,
`episode_id` int(10) UNSIGNED DEFAULT NULL,
`season` int(10) UNSIGNED DEFAULT NULL,
`episode` int(10) UNSIGNED DEFAULT NULL,
`source` varchar(191) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'local',
`negative_votes` int(10) UNSIGNED NOT NULL DEFAULT '0',
`positive_votes` int(10) UNSIGNED NOT NULL DEFAULT '0',
`reports` int(10) UNSIGNED NOT NULL DEFAULT '0',
`approved` int(10) UNSIGNED NOT NULL DEFAULT '1',
`order` int(10) UNSIGNED NOT NULL DEFAULT '0',
`created_at` timestamp NULL DEFAULT NULL,
`updated_at` timestamp NULL DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
INSERT INTO `videos` (`id`, `name`, `thumbnail`, `url`, `type`, `quality`, `title_id`, `episode_id`, `season`, `episode`, `source`, `negative_votes`, `positive_votes`, `reports`, `approved`, `order`, `created_at`, `updated_at`) VALUES
(33276636, '30-Ingles <img src=\'/[login to view URL]\'>', NULL, '[login to view URL]', 'external', 'Hd-Tv', 87982, 797195, 8, 2, 'local', 0, 0, 0, 1, 0, '2019-05-16 22:25:52', '2019-05-16 22:25:52'),
ALTER TABLE `videos`
ADD PRIMARY KEY (`id`),
ADD UNIQUE KEY `videos_url_title_id_unique` (`url`,`title_id`),
ADD KEY `videos_title_id_index` (`title_id`),
ADD KEY `videos_episode_id_index` (`episode_id`),
ADD KEY `videos_season_index` (`season`),
ADD KEY `videos_episode_index` (`episode`),
ADD KEY `videos_source_index` (`source`),
ADD KEY `videos_order_index` (`order`);
24 pekerja bebas membida secara purata $480 untuk pekerjaan ini
I will make the script to find multiple 404 pages. I understand why the problem is coming. I have few queries . Can you answer them if you are online/ Let us discuss
Hello there. I can write the script to find soft 404 external pages and as per the requirements given in the project. Please send me a message to discuss more and get started. Thanks
Hi, Dear How are you doing? I am very interested in your project. I am always ready for you. I wish you contact me as soon as possible. Let us discuss your project on chat in detail. Thanks for your regards.