Scrapes Automatic WordPress Scraper And Content Crawler Plugin
Automatic WordPress Scraper and Content Crawler Plugin …
Scrapes
The most advanced WordPress scraper and content crawler plugin to scrape content from any website automatically with visual selector.
Try live demo
View pricing
Get your content ready in minutes.
The easy to use interface of our automatic WordPress content crawler plugin is designed for the best user experience. Set up easily in a few steps and leave it in the background to get content from source sites.
Enter target URL to scrape
Inspect with visual selector
Run WordPress autoblog plugin
One WordPress autoblog plugin for all sources.
What Scrapes can do is completely limited to your imagination. Use the default settings for popular sources or create new tasks from scratch and crawl through almost all websites or use it as a RSS aggregator plugin for WordPress.
Usecase ideas.
News
Currencies
Weather forecasts
Live sport scores
Blogs & Magazines
Articles
Bulletin boards
Social profiles
Design inspirations
Images & Galleries
Videos
Movies & Celebrities
Music & Playlists
Apps & Games
Books
Themes & Plugins
Products
Freelance jobs
Syllabi
Restaurants
Recipes
Theaters
Airports & Flights
Transportation hours
Hotels
+ more
All unique and powerful features.
We have identified all the features that an automatic WordPress scraper plugin must have, and developed the state of the art Scrapes to meet all needs from the simplest to most advanced.
Instant control with detailed dashboard.
View all of your tasks on a single screen and easily manage them as you wish.
Supports all WordPress fields.
Scrapes automatically fills all fields those are supported, as if you had manually filled them out. Results of automatic WordPress content crawler plugin is no different from native one.
Post type
Taxonomies
Title
Content
Excerpt
Custom fields
Discussion
Author
Status
Date
Image
Visual selector.
You do not need any programming skills to make the necessary settings. All you need is to match the parts you need on the target page and the corresponding WordPress field with the visual selector.
Auto detect.
Content, featured image, next page etc. parts are matched with the corresponding fields automatically with automatic WordPress scraper plugin from the suitable source sites, so the most accurate section is provided for scraping with minimum effort.
RSS aggregator plugin for WordPress.
Get up to date subjects from feeds of websites which are not suitable for plain HTML scraping. Meet both your WordPress auto post plugin and RSS aggregator plugin for WordPress needs.
Advanced content customization.
You do not have to scrape the content with automatic WordPress content crawler plugin as is. Improve the layout automatically according to your needs before adding it to your own site.
Templates.
Specify the order of scraped data you want to appear on your site with the template option and personalize the layout of your post. All the options you can use are available as a template tag in the interface.
Usecase ideas: Add the origin site’s address which you get with WordPress auto post plugin to the end of the body, combine the different parts, and edit the layout using visual composer tags.
Find & Replace.
Remove specific words and phrases from the scraped text with the regular expression-supported find & replace feature, or replace them with your own words. Moreover, you can add an unlimited number of rules to run respectively.
Usecase ideas: Remove or replace links in the page, publish only a few words or sentences, remove ads with automatic WordPress content crawler plugin.
Math calculation.
Perform mathematical operations such as addition, subtraction, multiplication, and division on scraped data and obtain a new number. Combine numbers in different custom fields and create new formulas via automatic WordPress scraper plugin.
Usecase ideas: Include your own commission at the price before publishing a WooCommerce product.
Language translation.
Automatically translate content scraped in a specific language to one of dozens of different language options to get completely new content with Bing Microsoft Translator, DeepL Translator, Google Translate or Yandex Translate service.
Usecase ideas: Translate an English website published in another country to your local language with WordPress autoblog plugin.
Content spinner.
You can benefit from integrated ready to use 3rd party spinner services, or use the existing spinner plugins to alter scraped content.
Usecase ideas: Use The Best Spinner service with our RSS aggregator plugin for WordPress to differentiate the content.
Filters.
Filter the content in the source site to match the rules you specify. So, just make sure that the content is successfully passed through these filters is added to your own site. Moreover, you can add an unlimited number of rules.
Usecase ideas: Through our WordPress auto post plugin, pass posts with a specific word in title, just scrape content after a certain date, skip posts without a featured image, just scrape products under a certain price.
Integrated with themes and plugins.
If you just need more than automatic WordPress scraper plugin, it is for you. Running in compatibility with other themes and plugins is just one of the features that make it strong.
Works with WooCommerce.
From the online stores, you can use the custom post type and custom fields support in your WooCommerce store to collect content in the form of products. Earn revenue by selling products either in your own store or redirecting them to the target site through an affiliate program.
Product type
Regular price
Sale price
Product URL
Button text
SKU
Manage stock
Stock
Backorders
Stock status
Purchase note
Visibility
Featured
Gallery
Superfast and high quality support.
You receive top notch support with dedicated team of fastest professionals across the world.
Trusted by the community.
55. 000+ Satisfied customers can’t be wrong.
No additional costs or hidden fees, one time payment only.
Thanks to our transparent price policy and automatic activation system, you can get started with the download link and license code to be sent to your e-mail address after you complete the payment process.
Standard
$29/ per license
$58
50% OFF Limited time offer!
Lifetime version updates
Lifetime technical support
Instant download and use
OFFER GET EXTRA 50% OFF
On 5+ Scrapes purchases.
OFFER GET EXTRA 60% OFF
On 10+ Scrapes purchases.
OFFER GET EXTRA 75% OFF
On 30+ Scrapes purchases.
F. A. Q
We have answered and gathered all the questions you might have about Scrapes in detail.
Scrapes is designed to provide maximum performance even in the minimum system configuration provided by shared hosting. However, issues such as the hardware specification of your server, the connection speed of your server and the source site, some limitations defined on your server, how many tasks you are running at the same time etc. are factors that affect performance. The minimum requirements are listed below.
PHP 5. 2. 4+
dom, mbstring, iconv, json and simplexml extensions for PHP
Configurable maximum execution time for PHP
WordPress 3. 5+
Google Chrome, Firefox, Safari or Microsoft Edge for admin panel
You can already copy the content of any site to your own WordPress site manually without any tools. Scrapes is simply an automatic WordPress scraper plugin that allows you to automate this process. In other words, this question is not related to Scrapes, and there is no clear answer to “Yes” or “No”. Actually the answer differs according to the source site and your intended use of the content.
Some aim to reach more users by sharing their content with content sharing systems such as RSS (Rich Site Summary) or API (Application Programming Interface).
Some allow you to publish some of their content, ask you to give them the backlink to view the rest and target more hits.
Some in exchange for publishing their products on your site, pay you under their affiliate marketing programs and aim to sell more.
Some allow you to publish their content except commercial use.
Some refer to extra protection methods in order not to scrape their contents.
Although it is entirely your responsibility to copy the content of any website, our recommendations for not doing something illegal and avoiding Google penalties of duplicate content are as follows.
In necessary situation, you can contact the owner and request permission to publish the content in your site.
Using an SEO plug-in that allows you to define a Canonical URL, you can specify which source is used.
Change content to a different language with the translate feature, or spin and retrieve a new content with content spinner services.
With the template feature, you can add credit information or the backlink directed to the origin URL to the end of the content in WordPress auto post plugin mode for example.
You can publish a portion of the content by shortening it with Find & Replace feature instead of publishing the entire content.
You can extract content in “Draft” status and publish it after making changes.
Scrapes is designed to run compatible with other WordPress plugins and themes however some themes and plugins may require some complex string type values called array like a:1:{i:20041001103319;s:4:”test”;} which is not supported by our product.
In order to check your theme or plugin needs this type of strings, you can install free “Post Meta Inspector” plugin from. You will see an additional meta box on your posts / custom post types at the end of page like this screenshot after the installation. If every meta field has plain text values then you can easily use Scrapes to fill every data you need from the target website, otherwise you will need some manual edits after the scrape.
If the “Remember me” option is checked when logging in, the login information is stored in the browser as a cookie in encrypted form. You can extract content from resource sites which require logging, or using Cloudflare by defining these cookie values to Scrapes.
Scrapes can extract content from almost all sites and RSS feeds, except as noted type of sites below. You can find out whether the target you are planning to scrape is appropriate, by trying out the live demo of our product or by contacting us.
The content should be loaded when the page is opened. JavaScript / Ajax technologies and post-loaded page content are not supported.
On the listing page, there should be a “Pagination” or “Next page” button that redirects to the next page. The content pages which are loading dynamic content without address change like “load more” button pressing or using infinite scrolls are partially supported.
On the listing page, there should be links in a given order and directed to the detail pages. Listing pages with links that are not positioned in a specific order, opening pages in a popup instead of link redirection, or loading content after page load action with JavaScript / Ajax technologies are not supported.
The detail pages that are redirected from the listing page should have the same HTML page structure and must be a valid HTML. Non-valid detail pages with different HTML page structure may not be supported.
RSS and Atom feeds with non-valid XML structure may not be supported by our RSS aggregator plugin for WordPress.
There is no any limitation in our WordPress autoblog plugin to add an unlimited number of tasks, However each task may consume 10 – 30 MB of RAM during the process and they may slow down or not work properly if you have too many tasks running at the same time considering your server resources.
In order to reduce performance loss and prevent IP bans we suggest you to reduce your tasks or edit them to run in different timespans.
It can create “Simple”, “External” and “Virtual” WooCommerce products automatically from the target you want but “Grouped”, “Variable” and “Downloadable” product types and product attributes are not fully supported currently.
What Scrapes can do is completely limited to your imagination. You can create a site that will have rich content and get click, generate ad revenue, or open a store and earn commission revenue with affiliate marketing programs.
Scraper – Automatic Content Crawler Plugin for WordPress
Scraper is an automatic plugin that copies content and posts automatically from any web site. With tons of useful and unique features, Scraper WordPress plugin takes content creating process to another level.
Design your own scraping method with visual editor, Scraper will automatically fetch and post 7/24 for you.
Tutorials
We are creating step by step tutorials for plugin. Please let us know if you have any questions, we’ll be adding all details on tutorials.
Product
Troubleshoot
Settings
Other
Structures
Features
Scrape Any Website
With Xpath and regex methods, it’s possible to scrape any website!
Get Attributes
Scraper can parse element’s attributes, that means you could get links, image sources, video sources.
Set Feature Image
The plugin can extract any image to set as a featured image.
Duplicate Title Skip
Scraper verifies that there is no previous post with the same title.
Fix Encoding
The plugin automatically fix encoding errors and post it after fixing it.
Language Translation
Scraper automatically detects content and translate it to any language.
Support Categories
You could post your new content to any category that you’ve created.
Regex (JSON) Parser
You could parse an information that contains JSON objects.
Gallery Images
Scraper can parse any image sequence and create gallery with that.
Search & Replace
You could transform your content with search & replace feature.
Content Template with Variables
You could generate your unique content template with transform feature.
WooCommerce Products
Creating woocommerce products with Scraper is really easy. Scraper supports all tags of woocommerce.
Math Functions
You could sum, subtract or divide any number with math functions. It can be useful on price calculations.
Embed Any Post
Extract Youtube, Instagram or Vimeo ID from website and use it with embed transform.
Schedule Tasks
With different intervals you could schedule your tasks.
Post Update
You could select a post to update with scraping.
Link Strip
You could strip links from original post content.
Proxy Support
You could use proxies on scraping.
Cookie Support
Scraper supports cookie, you could define cookie for any task.
Post Status
You could set your post status to draft or publish.
Community Templates
There are hundreds of template on Scraper’s library. You could enable any of them easily.
F. A. Q.
+ Is there any feature to translate content?
Yes, it translates any content with Google Translate API. It supports all languages that supported by Google.
+ Is it possible to set featured image?
Yes, it’s possible. You could set any image element as a featured image.
+ Is it possible to create image galleries?
Yes, it’s possible. Scraper can detect image sequences and define them as gallery. It can also downloads image to your server.
+ Is it possible to create WooCommerce product?
Yes, we support all WooCommerce product custom fields and post types.
+ Is it possible to update prices on product?
Yes, with transform function, you could apply some regex expressions or mathematical functions for numbers. With this way, you could update numbers with yours or change the values with math functions.
+ Is it possible to scrape meta tags?
Yes, that’s possible. You could scrape HTML tags, meta tags, attributes or even JSON parts of page with visual editor.
+ Is it possible to JSON values on page?
Yes, that’s possible. You could write your own regex expression to extract data from JSON content.
+ Can i update test my task model before i create it?
Yes, in preview results section, you could test your model before you purchase or use plugin. It provides same results with WordPress plugin.
+ Can i use proxy for my content?
Yes, you could set proxy IP and port on settings button which is located beside fetch button.
+ Can i scrape all category with next page?
Yes, setting next page element is enough. It will automatically scrape all products or post items in a category page.
+ Can i get support for my task?
Yes, our support team will be asisting on your task or any issue that you experience on plugin.
+ Can i get refund?
Yes, if the plugin is not working as expected. Please reachout to use via before raising any refund request.
See Plugin’s Documentation
Visit Frequenly Asked Questions
Next Release
Changelog
04/30/2021 (v2. 0. 4)
– Bug: Fixed issue while adding more than 26 fields.
– Bug: Fixed Scheduled tasks get stuck in queue
– Bug: Fixed key name for new data field
– Added Cron URLs that can be run directly via server’s Cron Job
12/16/2020 (v2. 3)
– Bug: Fixed Duplicate Image download
– Bug: Fixed XML/RSS feed scrapping
– Feature: Added Task URLs that can be run directly via server’s Cron Job
– Feature: Added option to export output to excel directly without posting it to WP
– Added: Ready to use templates
– Removed Unnecessary Files & Folders
05/19/2020 (v2. 2)
– Bug: Author not set when posts imported via scheduled tasks
05/14/2020 (v2. 1)
04/27/2020 (v2. 0)
– Introducing Scraper Pro (for ajax scraping)
– Updated Documentation link
– Feature: Added support for `Alt` & `Title` for featured image
– Feature: Green Highlighter for Feed link selector
– Bug: Fixed warning issues
– Bug: Fixed Incorrect XML / RSS parsing message
– Bug: Fixed HTML Source scrapping issue
10/21/2019 (v1. 1. 3) + Bug Fixes
07/06/2019 (v1. 1) + UI Changes
+ Fixed: Scheduled Posts
+ Added: Ajax data parsing
+ Fixed: Parsing XML Feeds
+ Added: Non-english language support
+ Fixed: Local timezone capability
03/20/2019 (v1. 1) + UI revamped, many changes across the plugin and visual editor
+ Introducing Simplified view in “Scheduled Tasks” tab
+ Clone task feature introduced.
02/10/2019 (v1. 42) + ignore_params database issue has been fixed.
02/07/2019 (v1. 41)
+ product attributes issue has been fixed.
+ Split delimiter bug fixed for gallery images.
01/12/2019 (v1. 40)
+ DeepL Translation integration.
+ Disable URL parameters for URL uniqueness option.
+ Set featured image with post parent ID.
+ Total run limit functionality has been added.
+ Set author option has been added.
+ Set post slug option has been added.
12/06/2018 (v1. 39)
+ Multiple p element issue has been fixed for post_content.
+ Page increase stuck issues has been fixed.
11/27/2018 (v1. 38)
+ (Bugfix) Used images won’t be downloaded again.
+ Next page jump and auto-reset functions has been improved.
+ cURL gzip enabled.
+ WooCommerce product_cat and product_tag issues has been fixed.
11/13/2018 (v1. 37)
+ Bulk list URL problems has been fixed.
+ URL path issues has been fixed.
+ Next page trigger functions has been updated.
+ Gallery selection issues has been fixed.
+ Visual selector has been improved.
+ Default value for product attributes has been updated.
+ Next page selector has been improved.
+ Article content selector has been improved.
+ cURL feature has been added.
+ Proxy issues has been fixed.
+ Visual editor zoom issue has been fixed for low resolutions.
+ product_tag creation issue has been fixed.
+ Same variable name issue has been fixed for visual editor.
11/02/2018 (v1. 36)
+ URL Path issues has been fixed.
+ Bulk URL list option added.
+ Yandex Translation API has been added.
+ tbody issue has been fixed for Chrome.
+ Post formats has been added on project settings.
+ Ajax request error has been updated.
+ Index selection issue has been fixed for complex tables.
+ New track option added: no change on post status.
+ Notices removed from settings area.
10/22/2018 (v1. 35)
+ Pagination issues has been fixed.
+ Image download and is required issues has been fixed.
10/19/2018 (v1. 34)
+ Memory issue has been fixed.
10/19/2018 (v1. 33)
+ Aliexpress image issue has been fixed.
+ Error display added for timeout and memory issues.
+ Remove element for removing popups, preloaders.
+ “Required” field added. It pass if field is empty.
+ Special Conditions section added.
+ Task limit issue has been fixed.
10/16/2018 (v1. 32)
+ Next page function on preview results.
+ Auto-detection for scraping feature has been added.
+ Using variables as shortcodes on WordPress is possible now!
+ Visual editor Scraping base URL problem fixed.
+ Pause task issue has been fixed. (test edilecek)
+ Proxy URL issue on Visual tool, fixed.
+ Excluding posts by attributes is possible now.
+ Detailed logs for issues, problems.
+ Connection test functionality added.
+ Extract ID, HTML or URL from image attachment.
+ Limit task trigger function.
+ Next page URL issues has been fixed for location redirections.
10/12/2018 (v1. 31)
+ For is multiple field, JSON database format added.
+ Split content by words option added.
+ File download as attachment added. (pdf)
+ SKU Uniqueness method added.
+ _stock, _manage_stock, _backorders attributes has added.
+ Track changes function has been updated with uniquness method.
+ Post title can be used on filenames.
10/10/2018 (v1. 30)
+ Post excerpt field added, it’s also short product description.
+ Post delete issues has been fixed.
+ Post status change functionality added.
10/09/2018 (v1. 29)
+ Proxy issues has been fixed on visual editor
+ Visual editor’s local copy is on plugin with latest version.
+ Long URL issues has been fixed.
+ Post uniqueness issue has been fixed for post updates.
+ Delete post with source URL changes feature added.
+ Task process delay added.
+ Stop task feature replaced with pause task. It continues from last index.
+ decode feature added.
+ Multiple custom field feature added.
+ Media library attachment problem fixed for image objects.
+ Some UI problems has been fixed.
09/28/2018 (v1. 28)
+ Filename template feature added for all images
+ Strip and download images option added for content images
09/28/2018 (v1. 27)
+ Database migration issue has been fixed
09/28/2018 (v1. 26)
+ Gallery shortcode options added
+ Post URL and title uniqueness mechanism rewritten
+ Category creation issues has been fixed
+ Each post URL shortcode added
09/26/2018 (v1. 25)
+ Base URL issues for some web sites has been fixed (Aliexpress)
+ Clean Advertising codes improved
09/25/2018 (v1. 24)
+ Custom field issues has been fixed.
+ Documentation updated
+ Gallery images and featured images shouldn’t be duplicated
09/24/2018 (v1. 23)
+ Multiple category selection feature added
+ Filename transform section enabled
+ Base URL issues has been fixed for proxy
+ Custom taxonomy featured added
+ Gallery has been fixed for woocommerce products
+ Split function added for tags (With delimiter)
+ Google Translate API errors display
+ Serial selector for category, tags and images
+ Variable renderer minor fixes
+ Create new category with data field
+ Visual Editor improvements, UI bugs has been fixed
09/20/2018 (v1. 22)
+ Strip only some tags (for advertising codes)
+ Attributes UX fix
+ Base URL issues has been fixed
+ Post dates added for custom field
09/20/2018 (v1. 21)
+ Multiple attributes for Woocommerce products
+ UTF8 problems fixed for Non-English languages
09/19/2018 (v1. 20)
+ Preg match issues fixed
+ Xpath selector improvements
+ Auto detection feature for serial links
09/17/2018 (v1. 19)
+ Next page selection
+ Design issues and improvements
+ Result preview
+ Performance updates
+ Custom field issue fixed
+ Numeric functions problem fixed
09/15/2018 (v1. 18)
+ 0 byte image issue fixed
+ Image gallery issues fixed
+ Xpath selector fixed and improved
+ Regex parser method changed with /g flag
+ Find and replace method improved, it supports multiple find&replace
+ Excluding URLs is possible with this version
+ {{source_url}} option added
+ PHP errors fixed
09/07/2018 (v1. 16)
+ RSS feed or XML parsing method added.
+ Woocommerce product category issue fixed.
09/05/2018 (v1. 13)
+ Image Gallery Parse
+ UX Issue: single post checkbox lock
+ Strip only tags
+ A. I. Spinner Integration Error
+ Style Parser
09/05/2018 (v1. 12)
+ Trigger Issues Fixed
+ More Templates Added
+ Featured Image issues fixed
+ Cross-origin issues fixed
+ Xpath selector improved
08/17/2018 (v1. 0)
+ Cron issues fixed
+ Image download feature added
+ JSON Parsing Method Added
+ A. Spinner Service Enabled
+ Translation Service Enabled
+ First Release
Notes
On this section, we’ll be helping you about your concerns before you purchase.
Your hosting or server should allow incoming and outgoing network transfers for scraping.
Our support team can reach task’s content from service logs if you provide us any. This method makes easier support process and with this way our support team assist you quickly on any case.
We’ll be updating plugin for any bug or features that requested by users. Every update has been tested by our support team and softwares.
Plugin is licensed per domain. You should buy licence for each domain. You could move your licence to another domain.
Scraper doesn’t support ajax websites because of technical limitations.
Our plugin uses Google Translate official API unlike other plugins. So please make sure you have API key on that service before you purchase for translation services. Please also note that Google Translate’s Official API is a paid service.
Scraper’s plugin has functions that scrapes elements from given URLs. Our visual editor is a cloud based editor that hosted by us on our web service. Plugin doesn’t contains visual editor’s files in it. It only gets JSON data that built on visual editor.
Since Amazon blocked our servers, the plugin is unable to scrape any product from Amazon. That might be same for AliExpress and other big websites in future. Please make sure that plugin works properly before purchase. You could simply contact with our support team or comment and ask about support status. We’ll gladly test and inform you about changes.
You could request refund on any case that you are not satisfied on plugin.
Best WordPress Scraper Plugins – How To Automatically …
If you want to create a price comparison site or dropshipping store, WordPress scraper plugins can be very useful. Web scraping consists of gathering information from the web. That information is then organized or imported.
Some people consider scraping as an unethical or questionable activity. In actuality, web scraping can help you stay on top of changes. Price comparison sites can use scraped data to provide visitors with the most accurate information available.
There are plenty of WordPress scraping plugins available. In this post, I will mention some of the best WordPress content crawler plugins and their features so that you can choose the right tool for your needs.
Best WordPress Scraper PluginsOctolooks ScrapesExternal Importer ProWP Content CrawlerScraper – Content Crawler Plugin for WordPressCrawlomatic Multisite ScraperWP Scraper ProWordPress Automatic PluginWP RobotWordPress Scraper Plugins Conclusion
Best WordPress Scraper Plugins
Here are some of the best WordPress content scraper plugins you can use. Though they are paid options, all of them are packed with useful features.
Octolooks Scrapes
Octolooks Scrapes is the most advanced content crawler and WordPress scraper plugin by far. It uses a visual selector to scrap content from any site automatically. To work, you need to match the visual selector with the corresponding WordPress field on the target page. You don’t need any programming knowledge or expertise.
The plugin’s easy to use interface was created to provide the best possible user experience. The configuration is accomplished in only a few basic steps. You can leave it in the background, and information will be pulled from the source websites.
You can create new tasks for crawling or use the default settings. You can also use this plugin as a WordPress RSS aggregator plugin.
Scrapes automatically fills out all supported fields. The Octolooks WordPress scraper plugin will automatically match the next page, featured image, content, and other important information with the source websites’ corresponding fields.
You can use the template option to personalize post layouts and choose in what order the information you scrape will appear on your website.
The regular expression find & replace feature can remove certain words or phrases from the scraped text. You can also use your own words to replace them. There are no limits to the number of rules that you can run.
Subtraction, addition, division, multiplication, and other mathematical operations can be run. This WordPress content crawler plugin can create new formulas and combine numbers in different custom fields.
Yandex Translate, DeepL Translate, Bing Microsoft Translate, or Google Translate can automatically translate scraped content. Or you can translate WordPress site automatically using plugins like Weglot (check Weglot review) and WPML (see WPML review).
You can use one of the WordPress auto spinner plugins to change scraped content or let third-party spinner service like WordAi (see WordAi review) and Spin Rewriter (check Spin Rewriter review) do the work for you.
Information scraped from source websites can be filtered to ensure that it meets the set rules. Monitor the content to ensure that it successfully passes from the filters to your site.
Custom fields support and custom post type from your WooCommerce store can be used to scrape content in the form of products.
External Importer Pro
External Importer Pro plugin allows you to extract product data from eCommerce websites and import them into the WooCommerce site. No API access, CSV feeds, or XML is needed.
The plugin extracts complete product data directly from store sites. All you need to do is enter the specific listing or product URL. There are no bulky CSV files or API access to deal with. Product availability and prices are automatically updated. You can manage every aspect of the imported information.
Your existing affiliate IDS will automatically be used (if you added them via setting options) when creating affiliate links. You can even set dropshipping product margins if you want to import products for dropshipping purposes.
Features:
Automatic synchronization – Product availability and pricing information is automatically updated. Any products that are currently out of stock can be removed automatically. Updates are scheduled in the background so that they won’t interfere with any other operations.
Automatic import – Once new products appear on the target site’s listing page, they will also automatically be imported to your website. You’ll always have the most updated products in your store.
Unlimited products – The ability to import as many products as you want. You can import unlimited items from as many online store sites as you need.
Avoid getting blocked – The plugin will read and abide by cookie sessions, daily query quotas, random query intervals, real browsers’ headers, rules, user-agents rotation, requests throttling, etc., so that you don’t get blocked.
Use affiliate networks – Use deep links or dynamically change them to generate affiliate links.
Dropshipping features – You can create a dropshipping store, and items can be added as “simple” WooCommerce products. Flexible rules can be set for price markups.
Local and global attributes – You get to determine the product specifications assigned as global attributes (or taxonomies). You can then implement various WooCommerce catalog filters and widgets.
External images by URL – The ability to display external images without saving them to a local media library. External source sites can be scraped to pull the featured galleries and images you want to show on your site. This will greatly reduce the amount of hard drive storage on your server.
Dynamic categories – Products with extracted category paths will be automatically imported to the corresponding category.
For more info about this content crawler plugin for WordPress, you can check my External Importer Pro review.
WP Content Crawler
WP Content Crawler plugin can automatically extract information from almost any site. It uses CSS selectors to find content. It uses the Visual Inspector tool that simplifies finding CSS selectors by clicking on the respective elements on the target sites.
Visual Inspector – Clicking on an element will identify the CSS selector for that element. You can also find alternate CSS selectors that could be used. You don’t have to leave your admin panel to accomplish these tasks.
Crawl posts (scrape, grab and save) – Once the post URLs have been defined, this WordPress content crawler will automatically crawl them in the background. This will occur after settings are configured.
Recrawl (update) posts – Posts can be recrawled automatically to ensure that you have the most up to date content. You can opt to ignore older posts, select your update interval, and limit the number of times a particular post can be updated.
Content templates – Shortcodes can be used to create a gallery, list item, title, post content, and excerpt templates. You can use the options box to create templates for all CSS selector values.
Paginated posts – Paginated posts can also be saved. You don’t have to limit your searches to single page posts anymore.
Custom general settings for each website – Custom general settings can be set for each post.
Save all images – You can save all images in the post’s content.
Save images as a gallery – Images found on a target page can be saved as a gallery.
Proxy options – If your IP doesn’t have access to a particular site, you can use one or more proxies to pull information from target sites.
Automatic translation – Amazon Translate API, Google Cloud Translation API, Microsoft Translator Text API, or Yandex Translate API can be used to translate posts automatically.
Automatic spinning – Spinning can rewrite crawled content automatically. This can help to increase your search engine rankings. The plugin offers integration with paid services like Turkce Spin API and Spin Rewriter API.
Save WooCommerce products – Attributes, advanced options, inventory, shipping, and product prices can be saved. Items can be saved as either external or simple products. You can also define items as virtual or create a downloadable file option.
Regular expressions – Regular expressions can be specified in your “find-replace” options. This makes it easier to find and replace anything. Modifiers and delimiters can also be implemented to refine searches further.
Save “alt” and “title” attributes – All “title” and “alt” attributes are automatically retrieved from the target site when you save images. Those attributes are then assigned to the respective saved images. Templates can be created to align with your search engine optimization strategies.
Manual crawling tool – You can enter various URLs to save more than one post at a time using the manual crawling utility. Category URLs can also be entered for the tool to obtain the appropriate post URLs. You can set the crawler to crawl different posts simultaneously.
Scraper – Content Crawler Plugin for WordPress
Scraper Content Crawler plugin for WordPress is a plugin that automatically copies content and post from any site. It takes content creation to another level with its unique features and functions.
Any website can be scraped – Using Regex and Xpath methods means that you can scrape any site you want.
You can scrape attributes – Scraper can also retrieve element attributes. That means you can get links, image sources, video sources.
Featured image – Any image can be extracted and set as the featured image.
Content spinner – The A. I. Spinner plugin is fully supported. You can use this plugin to create unique content.
Language translation – The scraper will automatically detect content, which can then be translated into whatever language you prefer.
Gallery images – Any image can be parsed. You can use those images to create image galleries.
WooCommerce products – All WooCommerce tags are also supported. This simplifies adding WooCommerce products to your store.
Mathematical calculations – Math functions can subtract, add, divide or multiply numbers. This may come in handy in price calculations.
Schedule tasks – You can assign tasks to be conducted at various intervals.
Strip links – Strip links from original post content.
Proxy support – You can use proxies for scraping purposes.
Crawlomatic Multisite Scraper
Crawlomatic Multisite Scraper plugin is a website crawling and scraping, post generator autoblogging plugin. You don’t need API’s to scrape content.
This plugin will crawl the URL (it will search all links on a page), visit and extract content from each crawled URL. The crawling process is customizable. You set the crawling depth, crawling rate, maximum crawled article count, crawl only links with specific class or ID, etc.
You can scrape content from almost every site. If the content is loaded using JavaScript, the plugin can be combined with PhantomJS to scrape JavaScript generated content.
The crawling of sitemaps is fully supported.
The visual content selector support.
You can paginate site crawling. Article crawling will resume on the next page of the target site.
You can import prices for all crawled products (for WooCommerce-compatible sites). Dropshipping prices are automatically adjusted accordingly.
You can raise the prices of imported items by a predefined number. You could also multiply the amount by a set number, which is a useful option for dropshippers.
Proxies can be used for crawling.
If you cannot direct crawl (if you’re blocked, for example), you can always crawl the particular page from the Google cache.
Google Translate is supported. You can choose the language you want your site’s articles to appear in.
Text spinners are also fully supported. You can change the text that’s generated automatically. Words can be changed with their synonyms if you prefer. SpinRewriter, The Best Spinner, TurkceSpin, WordAI, and others can be used.
Site scraping and crawling can be configured to respect the robots’ HTML headers of scraped pages and files of scraped sites.
Tags and post categories of products can be created automatically.
Website crawling and scraping can be used to embed DailyMotion, Flickr, IGN,, Vimeo, or YouTube videos.
WP Scraper Pro
WP Scraper Pro plugin can extract content from different websites that you can then import into your WordPress posts and pages. The multiple scrape feature allows you to scrape content from hundreds of sites simultaneously. This can be accomplished on your WordPress site using the visual interface.
Visual interfaces help you select data using either a predetermined setup or a single scrape. To select content, highlight and either select or deselect the information you want to import. This can eliminate any data blocks that are irrelevant or unnecessary.
Categories, images, tags, and titles will automate repeatedly based on the rules you assign or a preselection. It will also retrieve all URLs from the target sites that you select.
WordPress Automatic Plugin
WordPress Automatic plugin is a convenient tool that can automatically post to WordPress from almost any site. There are plenty of import selections.
Besides the usual articles, you can also import the following content: Amazon and Walmart products, YouTube, Vimeo, and DailyMotion videos, Flickr and Instagram images, eBay auctions, social media posts (tweets, pins, Reddit and Facebook posts), classifieds from Craigslist, iTunes content (such as songs, podcasts, apps, eBooks), SoundCloud songs, and even Envato items.
You can select the content source and apply filter options by tag, author, and category. This means that not all of the target information will be imported.
You get to choose the images, format, post template, type, and status that the plugin will fetch. There are also advanced translation and rewriting options. You can even automatically replace certain words that you don’t want to be displayed on your site.
You can set post statuses to either published or draft. Certain phrases or words can be excluded. You can also strip all links before publishing a post. Featured images can be automatically set.
Settings can be altered, so duplicate titles, non-English posts, and posts without any images are skipped. Custom fields are automatically added to posts, and multisites are supported.
WP Robot
WP Robot is an autoblogging and content curation plugin. It allows you to automatically create WordPress blog posts by scraping content from other sites. It drip-feeds information related to your particular specialty or niche. This ensures that you’ll always have the most current content.
When you enable content source, its settings tab will appear above.
More than 30 content sources are supported, and each content source is automated. They can be used in whatever combination you prefer to find quality content for your website. According to what you need, there are many ways in which this tool can assist.
WP Robot can pull content from e-Commerce sites if you’re looking to post products from Amazon, AliExpress, Etsy, etc. The plugin can pull images from Flickr and Pixabay, songs from iTunes, YouTube, Vimeo videos, etc.
Commission Junction and Linkshare are some of the affiliate networks that WP Robot supports, and you can automatically post offers from them. RSS feed content can also be added to your site. If you want more than what the existing modules provide, this can give you some added freedom. For more info, be sure to check my WP Robot review.
WordPress Scraper Plugins Conclusion
Web scraping (also known as web harvesting, web data extraction, and screen scraping) acquires vast information from various sites. This data is then saved to another website or a database. Many web scraping solutions require additional knowledge and can be rather complicated. Using mentioned WordPress scraper plugins, content scraping is very easy.
If you want to create an affiliate store, price comparison site, deal site, or dropshipping store, you will need to add products to your site. It is better way to automate that process instead of manually adding products.
For that purpose, you will need a good plugin for importing products. While there are many solutions available, most of them require that you have a feed or API which will be used to import products.
But what if you don’t have a feed? What then? How to import products to the site if you don’t have access to feeds? In this case, you will need a WordPress web scraper plugin.
DISCLOSURE: Posts may contain affiliate links. If you buy something through one of those links, I might get a small commission, without any extra cost to you. Read more about it here.
